-- Logs begin at Mon 2022-08-08 13:48:09 UTC, end at Mon 2022-08-08 14:46:27 UTC. -- Aug 08 14:17:01.087851 dv-cinder-852009-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 14:17:01.100215 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 14:17:01 2022] *** Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: nodename: dv-cinder-852009-2-iscsi Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: machine: x86_64 Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: clock source: unix Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: pcre jit disabled Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: detected number of CPU cores: 4 Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: current working directory: / Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: detected binary path: /usr/bin/uwsgi-core Aug 08 14:17:01.102193 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: your processes number limit is 47680 Aug 08 14:17:01.102863 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: your memory page size is 4096 bytes Aug 08 14:17:01.102863 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: detected max file descriptor number: 2048 Aug 08 14:17:01.102863 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: lock engine: pthread robust mutexes Aug 08 14:17:01.102863 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: thunder lock: enabled Aug 08 14:17:01.102863 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 14:17:01.102863 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 14:17:01.141716 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: Python main interpreter initialized at 0x5645bae239c0 Aug 08 14:17:01.141716 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: python threads support enabled Aug 08 14:17:01.141716 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: your server socket listen backlog is limited to 100 connections Aug 08 14:17:01.141716 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: your mercy for graceful operations on workers is 90 seconds Aug 08 14:17:01.142041 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 14:17:01.142115 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: *** Operational MODE: preforking *** Aug 08 14:17:01.142341 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: *** uWSGI is running in multiple interpreter mode *** Aug 08 14:17:01.142341 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: spawned uWSGI master process (pid: 80122) Aug 08 14:17:01.142527 dv-cinder-852009-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 14:17:01.147255 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: spawned uWSGI worker 1 (pid: 80133, cores: 1) Aug 08 14:17:01.147255 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: spawned uWSGI worker 2 (pid: 80134, cores: 1) Aug 08 14:17:01.147255 dv-cinder-852009-2-iscsi devstack@keystone.service[80122]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 14:17:06.661386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 14:17:06.661386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: from cryptography.utils import int_from_bytes Aug 08 14:17:06.662242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 14:17:06.662242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: from cryptography.utils import int_from_bytes Aug 08 14:17:06.662835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 14:17:06.662835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: from cryptography.utils import int_from_bytes Aug 08 14:17:06.663333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 14:17:06.663333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: from cryptography.utils import int_from_bytes Aug 08 14:17:06.711936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 14:17:06.712113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 14:17:06.712274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] command line args: [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 14:17:06.712543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 14:17:06.712724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ================================================================================ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 14:17:06.712980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] admin_token = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.713228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config_dir = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.713417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.713605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] config_source = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.713786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] debug = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.713912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 14:17:06.713992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.714249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 14:17:06.714313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] command line args: [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 14:17:06.714369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.714530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.714646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 14:17:06.714783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.714867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ================================================================================ {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 14:17:06.715034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] admin_token = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_config_append = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] config_dir = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.715868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_dir = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] config_source = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] debug = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.716990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.717927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_config_append = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_dir = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_file = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.718830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.719943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] publish_errors = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.720989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] standard_threads = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.721902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_journal = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_json = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_stderr = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] use_syslog = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.722901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.723012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.723154 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.723389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.723657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.723827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.723946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.724058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] publish_errors = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.724176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.724300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.724421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.724557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.724681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.724795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.724906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.725057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.725180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.725300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.725415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.725547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.725686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] standard_threads = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.725812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.725993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.726159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.726281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.726508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.726633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.726755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.726869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.726980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] use_journal = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.727119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.727244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] use_json = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.727359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.727482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] use_stderr = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.727601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.727713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] use_syslog = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.727825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.727942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 14:17:06.728054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.728282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.728407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.728602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.728747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.728961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.729105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.729338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.729465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.729635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.729751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.729969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.730963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.731940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.732972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.733998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.734961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.735916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.736968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.737088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.737202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.737313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.737425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.737536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.738946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.739981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.740969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.741890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.742929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.743918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.744984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.745908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.746994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.747932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.748934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.external = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.749964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.password = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] auth.token = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.750888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.751979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.752927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.753041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.753154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.753267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.754905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.external = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.755993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.password = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] auth.token = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.756956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.757932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.758964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.759973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.760905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.761494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.762970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.763921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.764847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765175 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.765982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.766958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.767940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.768888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.769929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.770946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.771218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.771468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.771620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.771739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.771865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.771983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.772936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.773982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.774899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.775836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.776908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.777948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.778987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.779937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.780979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.781899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.782974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.783962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.784980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.785983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.786982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.787940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.788965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.789997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] role.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] role.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.790908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.791964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.792896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.793920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.794958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.795960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796419 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.796918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797269 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.797960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80133) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 14:17:06.798522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.798846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.799054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.799322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.799589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.799839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.800590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.800847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.801089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.801336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.801573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.801811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.802044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.802292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.802541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.802770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.803011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.803328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.803591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.connection = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.803837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.804083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.804320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.804559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.804794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.805030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.805263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.805497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.805738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.805972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.806215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.806447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.806692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.806925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.807200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.807433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.807675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.807907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 14:17:06.808076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80134) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 14:17:06.828318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.828551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.828779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.833348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.833579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.833802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.838657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.843537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.867392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.867625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.870178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.870412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.920569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.921439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.971964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.972198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.972964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:06.973208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.037640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.038933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.039182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.039405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.039623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.039838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.062961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.063317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.063452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.063536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.063671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.063751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.063887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.064102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.074752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.074960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.087456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.087674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.087758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.087898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.087980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80133) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.088970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80134) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 14:17:07.106014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.106225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 14:17:07.106532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.106613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.106739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 14:17:07.106835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 14:17:07.107032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.107135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.107295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 14:17:07.107374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.107574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.107664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.107832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.107910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.108089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.108344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.119727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.120009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.120283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.120364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.120570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.120665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.120864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.121148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.127474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.128036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.130760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.131045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.131154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.131360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.131440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.131649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.131743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.131927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.132837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.133873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.156613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.156810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.158122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.158303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.158411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.158588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.163558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.163722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.163847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.164006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.164136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.164291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.170745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.170834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.170991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.171072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.175029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.175749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.175887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.175974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.176171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.176270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.176456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.176536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.176726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.176806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.177002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.177093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.177332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.177428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.177545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.177625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.193477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.193576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.193755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.193838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.197727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.197837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.200891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.200989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.201193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.201287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.205555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.205652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.208765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.208861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.211931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.212910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.213099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.213179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.213376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.213469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.229783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.229877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.230079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.230160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.230351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.230445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.230614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.230695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.237580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.237706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.239963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.240050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.240151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.240223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.242794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.242918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.243097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.243291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.243420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.243544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.243715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.243886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.244011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.244168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.256503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.256791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.256909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.257073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.257193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.257354 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.257482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.257639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.257770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.257917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.258056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.258194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.258340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.258622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.274795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.275233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.277772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.278210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.280727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.281184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.282307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.282756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.283833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.284115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.284281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.284903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.285085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.285370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.291326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.291807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.292930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.293418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.295882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.296174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.296385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.296466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.296680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.296769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.296965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.297246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.305321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.305614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.305889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.305970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.306184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.306468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.312752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80134) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.313318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.313601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.313694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80133) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 14:17:07.313879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.313960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.314986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.315075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.315324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.315405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.315611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.315718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.315943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80134) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.316846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.317086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.317367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80133) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 14:17:07.350364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5645bae239c0 pid: 80134 (default app) Aug 08 14:17:07.351000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5645bae239c0 pid: 80133 (default app) Aug 08 14:17:07.353699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbf3a9-c063-49bf-ab2e-61c53ee36bd0 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:07.353979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbf3a9-c063-49bf-ab2e-61c53ee36bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:07.354285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbf3a9-c063-49bf-ab2e-61c53ee36bd0 None None] PATH_INFO: `/v3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:07.358460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Mon Aug 8 14:17:07 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:14.948637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:14.949203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:14.949203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:14.953547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:14.959223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:14.959665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:14.959756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:15.031664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: INFO dbcounter [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] Registered counter for database keystone Aug 08 14:17:15.051975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG oslo_db.sqlalchemy.engines [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80134) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 14:17:15.058100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writer thread running {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 14:17:15.247071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG passlib.handlers.bcrypt [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80134) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 14:17:15.247579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG passlib.handlers.bcrypt [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80134) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 14:17:15.302290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:15.303917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:15.445375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 08 14:17:15.589503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:15.658322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: INFO dbcounter [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] Registered counter for database keystone Aug 08 14:17:15.679759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG oslo_db.sqlalchemy.engines [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80133) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 14:17:15.685911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writer thread running {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 14:17:15.805491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5678e7-a1a2-4970-b2ce-c05bea737de0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:15.809810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:15.810448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:15.810732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:15.811007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:15.820796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:15.872586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 14:17:15.872870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534285e-5351-4e88-bb5c-c698d84bd5d7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:15.883780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Mon Aug 8 14:17:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:17:15.889702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:15.904247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087d838-452c-420b-980f-686b0ebfc828 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:15.908761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:15.909880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:15.910168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:15.910443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] PATH_INFO: `/v3/projects/20988f0d51544772b78672526cf109cb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:15.911914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] RBAC: Authorizing `identity:get_project(project_id=20988f0d51544772b78672526cf109cb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:15.962227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 14:17:15.962762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bdc8a-d8cf-470d-ac3f-6e4f1e689991 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:15.967163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Mon Aug 8 14:17:15 2022] GET /identity/v3/projects/20988f0d51544772b78672526cf109cb => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:17.948663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506573-777f-4321-a904-45a8aa03f910 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:17.949173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506573-777f-4321-a904-45a8aa03f910 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:17.949277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506573-777f-4321-a904-45a8aa03f910 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:17.953265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:17.959181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:17.959579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:17.959722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:17.981422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:17.982920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:17.990969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:17 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 14:17:18.012783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506573-777f-4321-a904-45a8aa03f910 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:18.025238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506573-777f-4321-a904-45a8aa03f910 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:18.029417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fa1b2-1159-47db-9bcf-81412ea83a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:18.030010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fa1b2-1159-47db-9bcf-81412ea83a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:18.030289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fa1b2-1159-47db-9bcf-81412ea83a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:18.030560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fa1b2-1159-47db-9bcf-81412ea83a59 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:18.038678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fa1b2-1159-47db-9bcf-81412ea83a59 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:18.040486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fa1b2-1159-47db-9bcf-81412ea83a59 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:18.049159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Mon Aug 8 14:17:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:17:18.054860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:18.064158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697386f8-a646-4098-9a44-90cb1fa44d7a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:18.068525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd22446-5f0a-4bee-adb0-2f2978ea7620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:18.069176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22446-5f0a-4bee-adb0-2f2978ea7620 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:18.069458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22446-5f0a-4bee-adb0-2f2978ea7620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:18.069735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22446-5f0a-4bee-adb0-2f2978ea7620 admin admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:18.071060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd22446-5f0a-4bee-adb0-2f2978ea7620 admin admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:18.072896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd22446-5f0a-4bee-adb0-2f2978ea7620 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:18.074408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Mon Aug 8 14:17:18 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:20.121526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990943b-dff4-41d3-b354-93728bfcae38 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.122054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990943b-dff4-41d3-b354-93728bfcae38 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.122162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990943b-dff4-41d3-b354-93728bfcae38 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.123554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:20.128773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.129203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.129295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.147370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:20.148521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:20.158707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9103c9-bc91-4db9-8759-e6f7121c2794 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.159584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9103c9-bc91-4db9-8759-e6f7121c2794 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.159853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9103c9-bc91-4db9-8759-e6f7121c2794 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.161155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:20.166414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.166707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.166990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.210313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG passlib.handlers.bcrypt [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80133) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 14:17:20.210757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG passlib.handlers.bcrypt [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80133) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 14:17:20.229776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:20.252181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.256468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:20.258103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:20.275550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:20.356445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd18c5d1-2663-4dd0-90f3-bb7713b54409 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:20.357610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:20.379481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.389873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:20.390512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.390796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.391072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.402970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:20.412281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:20.413261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:20.414078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:20.420002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.484863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efa273-8a9a-41ea-8746-4939b5695f71 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:20.487588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:20.501531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:20.521538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcdf4f-60e9-4027-b505-f60082a565c1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:20.526015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:20.526920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.527252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.527517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.556064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:20.556427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.556708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.556983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.557661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:20.557960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:20.559415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:20.571812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:20.577573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.597582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:20.599128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:20.707607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 8/16] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:17:20.713509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.716718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:20.794183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33e55d-a68b-40db-8bea-a14ca9867dc4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:20.820826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0a76c-cf37-40de-ba02-ff7ac9c3ead2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:20.829766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:20.830424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.831337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.831616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.832766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:20.834238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:20.835854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:20.841208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.854534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:20.855447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.855730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.856002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.856130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:20.878080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:20.879133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:17:20.898392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:20.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:17:20.900120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:17:20.900120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:17:20.900120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:20.900120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 9/18] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:20.905440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:20.946371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ec286-6449-4acb-8b5d-a7d67fd1c9c3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:20.962405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:20.980587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:20.981508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:20.981790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:20.982063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:20.983202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:20.984671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:20.986211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 10/19] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:20.992600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:21.007289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f9645-b58d-4d61-99b0-aac29ea3ef69 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:21.041553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:21.042159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:21.042440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:21.042710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:21.043555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:21.043863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:21.045250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c11a3-9def-4a72-8dac-527d3839701d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:21.063928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 10/20] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:21.077135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:21.098198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e18d2-77f8-464d-8245-93e90ed5a25a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:21.135266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:21.135960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:21.136166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:21.136445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:21.137193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.domain.id=default, user.id=a4f3ff67989b433e88e7bd1a0058bb7d) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:21.137505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:21.138991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e7cca-fa62-4616-ab3a-0401365a8c3e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:21.147469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 11/21] 10.222.0.50 () {56 vars in 1322 bytes} [Mon Aug 8 14:17:20 2022] GET /identity/v3/role_assignments?user.id=a4f3ff67989b433e88e7bd1a0058bb7d&scope.domain.id=default&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:23.468842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e950e-7bf0-4f05-a8c1-366d5110f2ba None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.469849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e950e-7bf0-4f05-a8c1-366d5110f2ba None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.470330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e950e-7bf0-4f05-a8c1-366d5110f2ba None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.471963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 11/22] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:23.484396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b52be-1b93-46be-9439-bad9f79777b9 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.485230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b52be-1b93-46be-9439-bad9f79777b9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.485748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b52be-1b93-46be-9439-bad9f79777b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.512131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d38b52be-1b93-46be-9439-bad9f79777b9 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:23.513349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d38b52be-1b93-46be-9439-bad9f79777b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:23.595759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 12/23] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:23.631178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e950e-7bf0-4f05-a8c1-366d5110f2ba None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:23.691268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60190-5e13-4a64-a102-68e4070192bd None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.691551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60190-5e13-4a64-a102-68e4070192bd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.692139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f60190-5e13-4a64-a102-68e4070192bd None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.694566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 13/24] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:23.699822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa291-85e4-42ed-a101-d841fd4629ab None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.700766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa291-85e4-42ed-a101-d841fd4629ab None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.701244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06afa291-85e4-42ed-a101-d841fd4629ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.732537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-06afa291-85e4-42ed-a101-d841fd4629ab None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:23.734459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-06afa291-85e4-42ed-a101-d841fd4629ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:23.809207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e950e-7bf0-4f05-a8c1-366d5110f2ba None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:23.840251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:23.868262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 14/25] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:23.869530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dd192-97f1-48d6-982b-4f1bc41e7ede None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.869818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dd192-97f1-48d6-982b-4f1bc41e7ede None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.870097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dd192-97f1-48d6-982b-4f1bc41e7ede None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.871676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 15/26] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:23.876987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:23.878172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.878423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.878694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.880043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:23.880470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:23.880882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:23.888642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:23.902343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:23.903804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:23.904923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:23.906560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:17:23.933452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:23.934591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:17:23.935717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:17:23.935717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:17:23.935717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:23.935717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 12/27] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:23.935717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:23.988441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 16/28] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:23.989391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:24.082259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f977ef2-6c79-4ada-a219-e9dc49dd2e80 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:24.094301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf17b1-4043-4174-9625-a35cbbcca5bb None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:24.134351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:24.135415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.135925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.136389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.137263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:24.137772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:24.139446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4ae15-4f28-4bc5-9e43-b6aa0eb3d8f0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:24.141717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:24.161877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 17/29] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:24.163637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244e23e-50b4-42e5-ae2d-8ad544d19631 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.164141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244e23e-50b4-42e5-ae2d-8ad544d19631 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.164679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244e23e-50b4-42e5-ae2d-8ad544d19631 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.166311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 18/30] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:24.171600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244e23e-50b4-42e5-ae2d-8ad544d19631 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:24.232791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:24.233443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.233932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.234372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.235360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:24.237151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:24.285551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244e23e-50b4-42e5-ae2d-8ad544d19631 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:24.321470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad8aef-b001-4e15-81d6-423cada709a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:24.322447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8aef-b001-4e15-81d6-423cada709a3 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.322921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8aef-b001-4e15-81d6-423cada709a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.323417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad8aef-b001-4e15-81d6-423cada709a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.324278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad8aef-b001-4e15-81d6-423cada709a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:24.325888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad8aef-b001-4e15-81d6-423cada709a3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:24.369965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 13/31] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 14:17:23 2022] POST /identity/v3/roles => generated 219 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:24.370706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:24.411472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:24.472195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 19/32] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 14:17:24 2022] POST /identity/v3/roles => generated 225 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:24.474201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.474558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.475009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.484156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c08b1f-6238-45e7-aaca-f6c0f37c34e5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:24.514672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:24.520211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:24.524612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:24.525266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.525557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.525835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.526789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:24.528515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:24.569257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 14/33] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:17:24 2022] POST /identity/v3/projects => generated 301 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:24.617584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:24.648837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 20/34] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:24.696878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:24.803799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:24.818633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e6a95-0fd3-4e8f-bf68-bd50c8099757 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:24.864453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d3bed-3381-466c-a23b-43c005e7029d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:24.865046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d3bed-3381-466c-a23b-43c005e7029d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.865338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d3bed-3381-466c-a23b-43c005e7029d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.865738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d3bed-3381-466c-a23b-43c005e7029d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.880717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d3bed-3381-466c-a23b-43c005e7029d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-db0d3bed-3381-466c-a23b-43c005e7029d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:24.887649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:24.888514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:24.889298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 15/35] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:24.895514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:24.987235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80969e3-aee7-4b12-aee2-1f7ecfd0e936 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.987568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80969e3-aee7-4b12-aee2-1f7ecfd0e936 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.987829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80969e3-aee7-4b12-aee2-1f7ecfd0e936 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:24.989309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 16/36] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:24.994188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:24.994476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:24.994759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.010035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b53c2b-da01-4b64-9c3a-d9f2b2e7dba8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.013082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:25.014589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:25.051312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.052462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.052748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.053026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.053708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:25.054017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.055568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.065493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 21/37] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:25.070947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:25.085186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 17/38] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:25.107244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:25.124567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:25.179711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:25.180637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e1079-c9ae-4b36-8bc5-fd6ac356a54d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.210702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36b5c8-836e-4529-8975-ca6b98ee4c17 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.213956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e262792-071d-4813-af21-af3c965ba28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.214585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e262792-071d-4813-af21-af3c965ba28c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.214864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e262792-071d-4813-af21-af3c965ba28c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.215178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e262792-071d-4813-af21-af3c965ba28c None admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.216330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e262792-071d-4813-af21-af3c965ba28c None admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.217787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e262792-071d-4813-af21-af3c965ba28c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.219410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 22/39] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:25 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:25.220512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308ebee-7207-4059-a605-5bc240fbadb6 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.221351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308ebee-7207-4059-a605-5bc240fbadb6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.221623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308ebee-7207-4059-a605-5bc240fbadb6 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.222795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 23/40] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:25.225315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308ebee-7207-4059-a605-5bc240fbadb6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:25.251115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631d419-d995-46ed-b2e2-523c287cd2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.251689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631d419-d995-46ed-b2e2-523c287cd2b3 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.252001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631d419-d995-46ed-b2e2-523c287cd2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.252283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631d419-d995-46ed-b2e2-523c287cd2b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.252948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631d419-d995-46ed-b2e2-523c287cd2b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.254323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631d419-d995-46ed-b2e2-523c287cd2b3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.268118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 18/41] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 14:17:25 2022] POST /identity/v3/roles => generated 223 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:17:25.279299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.279299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.279299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.297081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:25.298320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:25.339537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308ebee-7207-4059-a605-5bc240fbadb6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.356274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:25.380627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 19/42] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:25.381642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.382313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.382601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.382881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.384131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.385632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.387301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 24/43] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:25.398002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:25.403819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:25.472301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:25.515680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36cbe07-fcd5-4f5d-8f55-e5dcb9b38fe0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.525895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d22f3-d5d9-42b0-8964-2f5442eca890 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.551764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c92802-a4d2-48a0-bd21-3cdc2e7f7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.552429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c92802-a4d2-48a0-bd21-3cdc2e7f7964 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.552712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c92802-a4d2-48a0-bd21-3cdc2e7f7964 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.552989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c92802-a4d2-48a0-bd21-3cdc2e7f7964 None admin] PATH_INFO: `/v3/domains/default/users/a4f3ff67989b433e88e7bd1a0058bb7d/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.555169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c92802-a4d2-48a0-bd21-3cdc2e7f7964 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a4f3ff67989b433e88e7bd1a0058bb7d, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.556627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c92802-a4d2-48a0-bd21-3cdc2e7f7964 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.559502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.560138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.560426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.560708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.562538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.564406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.566507 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 25/44] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:25.572594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:25.572828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 20/45] 10.222.0.50 () {58 vars in 1361 bytes} [Mon Aug 8 14:17:25 2022] PUT /identity/v3/domains/default/users/a4f3ff67989b433e88e7bd1a0058bb7d/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:17:25.630874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:25.693629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdc2c3-76fa-446e-832a-7c04138b6067 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:25.739238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72c5ec-3c38-4464-8666-a6ff7755f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:25.739865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72c5ec-3c38-4464-8666-a6ff7755f36e None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:25.740148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72c5ec-3c38-4464-8666-a6ff7755f36e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:25.740429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72c5ec-3c38-4464-8666-a6ff7755f36e None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:25.741090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72c5ec-3c38-4464-8666-a6ff7755f36e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:25.742560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72c5ec-3c38-4464-8666-a6ff7755f36e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:25.774471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 26/46] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:17:25 2022] POST /identity/v3/projects => generated 312 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:17:27.543778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f7958-1cbe-4167-b1da-e277155e2e65 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:27.544303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f7958-1cbe-4167-b1da-e277155e2e65 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:27.544385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f7958-1cbe-4167-b1da-e277155e2e65 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:27.545654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 21/47] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:27.550847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd228cb-52fd-4a63-94a4-1e340d0f7552 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:27.551266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd228cb-52fd-4a63-94a4-1e340d0f7552 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:27.551549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd228cb-52fd-4a63-94a4-1e340d0f7552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:27.570622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ecd228cb-52fd-4a63-94a4-1e340d0f7552 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:27.571829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ecd228cb-52fd-4a63-94a4-1e340d0f7552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:27.639335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 27/48] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:27.661914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f7958-1cbe-4167-b1da-e277155e2e65 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:27.765331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f7958-1cbe-4167-b1da-e277155e2e65 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:27.765781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f676031-7199-479e-bec6-5a8ad9a47026 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:27.766071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f676031-7199-479e-bec6-5a8ad9a47026 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:27.766349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f676031-7199-479e-bec6-5a8ad9a47026 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:27.767772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 28/49] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:27.772533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:27.772815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:27.773088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:27.785771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:27.790540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:27.791803 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:27.803046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:27.803977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:27.804266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:27.804709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:27.817653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:27.819939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:27.822757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:27.823679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:27.824410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 22/50] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:27.830331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:27.864322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 29/51] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:27.887286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:27.938438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868adbf-21a1-41cb-8225-6564a279c21d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:27.972147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:27.972764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:27.973049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:27.974030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:27.974030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:27.974566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:27.975754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:27.987571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500a7cf-d42d-4fa1-b902-1fd60d8b3549 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:27.988979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 23/52] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:27.994317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:28.021583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:28.022212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:28.022506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:28.022755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:28.023892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:28.025329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:28.026866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 30/53] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:28.032531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:28.037710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:28.063309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:28.094793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f16637-6ecc-456a-8d2d-521e700a5fc8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:28.128930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:28.129563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:28.129848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:28.130120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:28.131274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:28.131591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0c897-cb25-4a37-a67b-2ec4963af6bd None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:28.132724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:28.134296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 24/54] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:27 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:28.139666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:28.165664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:28.166279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:28.166563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:28.166840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:28.167529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:28.168967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:28.196782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 31/55] 10.222.0.50 () {60 vars in 1140 bytes} [Mon Aug 8 14:17:28 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:17:28.244187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153658e-42f3-4b41-8744-f29fea72d639 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:28.257996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:28.284794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ddcb6-1a6e-45d5-b057-dc365b4e537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:28.285444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddcb6-1a6e-45d5-b057-dc365b4e537d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:28.285732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddcb6-1a6e-45d5-b057-dc365b4e537d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:28.286011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ddcb6-1a6e-45d5-b057-dc365b4e537d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:28.287227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ddcb6-1a6e-45d5-b057-dc365b4e537d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:28.288752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ddcb6-1a6e-45d5-b057-dc365b4e537d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:28.290380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 25/56] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:28.296454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:28.385136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:28.404982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0fca-e41b-4d04-b3fd-cf2fd32ec8c0 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:28.440402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:28.441012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:28.441294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:28.441567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:28.442291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.domain.id=default, user.id=a4f3ff67989b433e88e7bd1a0058bb7d) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:28.442597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:28.444083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30894bd9-e296-4ee5-85ea-e28ffb39dbe7 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:28.449850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 32/57] 10.222.0.50 () {56 vars in 1322 bytes} [Mon Aug 8 14:17:28 2022] GET /identity/v3/role_assignments?user.id=a4f3ff67989b433e88e7bd1a0058bb7d&scope.domain.id=default&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:30.215673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b9aa-8b9e-4982-8dd7-31c0148dd56a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:30.216160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b9aa-8b9e-4982-8dd7-31c0148dd56a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:30.216160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55b9aa-8b9e-4982-8dd7-31c0148dd56a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:30.217491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 26/58] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:30.223285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:30.223710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:30.223836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:30.241315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:30.242492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:30.308775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 33/59] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:30.333872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b9aa-8b9e-4982-8dd7-31c0148dd56a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:30.432370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55b9aa-8b9e-4982-8dd7-31c0148dd56a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:30.466069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911151-138b-4d76-9967-bbb6393071e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:30.466627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911151-138b-4d76-9967-bbb6393071e6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:30.466911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911151-138b-4d76-9967-bbb6393071e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:30.467215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911151-138b-4d76-9967-bbb6393071e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:30.468314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911151-138b-4d76-9967-bbb6393071e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:30.469770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911151-138b-4d76-9967-bbb6393071e6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:30.471390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 27/60] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:30.476977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:30.550484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:30.573265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4d4f8-95bc-4497-a2ad-2ba2a9de07e4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:30.606452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463872df-2398-4c23-be8e-8e9ac6047431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:30.607053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463872df-2398-4c23-be8e-8e9ac6047431 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:30.607417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463872df-2398-4c23-be8e-8e9ac6047431 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:30.607628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463872df-2398-4c23-be8e-8e9ac6047431 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:30.608368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463872df-2398-4c23-be8e-8e9ac6047431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:30.609706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463872df-2398-4c23-be8e-8e9ac6047431 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:30.634827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 34/61] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:30 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:33.711937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6807-d3cd-4f12-a91e-94d672613b22 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:33.712451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6807-d3cd-4f12-a91e-94d672613b22 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:33.712523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc6807-d3cd-4f12-a91e-94d672613b22 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:33.713977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 28/62] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:33.719136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e45c3a-432f-438e-a823-3ba1943b5d1e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:33.719538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e45c3a-432f-438e-a823-3ba1943b5d1e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:33.719701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e45c3a-432f-438e-a823-3ba1943b5d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:33.744096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-44e45c3a-432f-438e-a823-3ba1943b5d1e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:33.745309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-44e45c3a-432f-438e-a823-3ba1943b5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:33.844498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 35/63] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:33.894822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc6807-d3cd-4f12-a91e-94d672613b22 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:33.964926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.009856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cac0f3-2d04-4686-ad6b-96429a020783 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.010499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cac0f3-2d04-4686-ad6b-96429a020783 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.010984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cac0f3-2d04-4686-ad6b-96429a020783 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.012833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 36/64] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:34.024542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaf5e3-fe01-44d4-90f3-7b2b14e401fa None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.025126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaf5e3-fe01-44d4-90f3-7b2b14e401fa None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.025626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beaf5e3-fe01-44d4-90f3-7b2b14e401fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.039819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc6807-d3cd-4f12-a91e-94d672613b22 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.064186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4beaf5e3-fe01-44d4-90f3-7b2b14e401fa None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:34.065905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4beaf5e3-fe01-44d4-90f3-7b2b14e401fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:34.078416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885a601-9217-4740-ac62-565e6f3b2c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.079189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885a601-9217-4740-ac62-565e6f3b2c2a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.079476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885a601-9217-4740-ac62-565e6f3b2c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.079748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885a601-9217-4740-ac62-565e6f3b2c2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.091241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885a601-9217-4740-ac62-565e6f3b2c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f885a601-9217-4740-ac62-565e6f3b2c2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:34.097042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:34.098067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:34.098923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 29/65] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:34.100798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fb3a8-6c4b-4e01-8cb7-ba34c6b9beba None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.101078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fb3a8-6c4b-4e01-8cb7-ba34c6b9beba None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.101361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12fb3a8-6c4b-4e01-8cb7-ba34c6b9beba None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.102806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 30/66] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:34.107561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fb3a8-6c4b-4e01-8cb7-ba34c6b9beba None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.141979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.155194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 37/67] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:34.156824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869cd5-bcbc-4e88-aed1-77da754aef85 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.157247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869cd5-bcbc-4e88-aed1-77da754aef85 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.157658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869cd5-bcbc-4e88-aed1-77da754aef85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.178265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f2869cd5-bcbc-4e88-aed1-77da754aef85 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:34.179884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f2869cd5-bcbc-4e88-aed1-77da754aef85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:34.249049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12fb3a8-6c4b-4e01-8cb7-ba34c6b9beba None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.252936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 38/68] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:34.254323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56579fa1-cdad-4a8e-84f7-801ab1962eaa None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.254614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56579fa1-cdad-4a8e-84f7-801ab1962eaa None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.254888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56579fa1-cdad-4a8e-84f7-801ab1962eaa None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.260040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 39/69] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:34.260040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56579fa1-cdad-4a8e-84f7-801ab1962eaa None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.263589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.296126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.297057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.297343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.297869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.298538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:34.299421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:34.300815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a62186-6054-48e3-b6ee-0340595a1cd6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:34.310409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 31/70] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:34.311985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.312274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.312551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.331677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:34.332892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:34.375809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56579fa1-cdad-4a8e-84f7-801ab1962eaa None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.398776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.399037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 32/71] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:34.399906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.419202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.420078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.420362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.420637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.431033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:34.435775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:34.436614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:34.437767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 40/72] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:34.439021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.496585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.500714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21945-8573-48ed-bea6-ccabd0ba28e0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.536530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13b190-a2c6-4bb6-8308-a0e24a6c703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.537155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b190-a2c6-4bb6-8308-a0e24a6c703a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.537438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b190-a2c6-4bb6-8308-a0e24a6c703a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.537718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13b190-a2c6-4bb6-8308-a0e24a6c703a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.540421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8159cb-5882-489f-b60c-be8ed53589e3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.547655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13b190-a2c6-4bb6-8308-a0e24a6c703a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-7e13b190-a2c6-4bb6-8308-a0e24a6c703a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:34.552318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:34.553178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 33/73] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:34.558993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abbe76-e55a-4aff-b1ca-502a45a27adb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.559701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abbe76-e55a-4aff-b1ca-502a45a27adb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.559701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abbe76-e55a-4aff-b1ca-502a45a27adb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.559701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 34/74] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:34.559701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abbe76-e55a-4aff-b1ca-502a45a27adb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.578182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.578817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.579110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.579404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.594348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:34.596792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:34.598459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 41/75] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:34.599675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.644652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.663212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abbe76-e55a-4aff-b1ca-502a45a27adb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.696995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.697627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.697910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.698192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.699431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:34.700827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:34.701794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d30c1-bbeb-4178-8b42-7633264de978 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.702423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 35/76] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:34.707802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.731724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.741523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.742239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.742385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.742672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.743386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:34.743693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:34.745078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feccbe-16cf-4e01-b163-6f10d7a3b854 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:34.754216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 42/77] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:34.755354 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.755640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.755916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.774984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:34.776238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:34.808374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aed83e8-eb58-423e-bafc-7e0ac0c0dae7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.844604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.845216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.845559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.845811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:34.846480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:34.846786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:34.847299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 43/78] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:34.848150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.848242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:34.868686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 36/79] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:34.872162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:34.881373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.950046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd53a0-8b48-4537-b2b7-5eac47b3bc88 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.966198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:34.978449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753206-1f8c-4018-ac1a-15abc041e772 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:34.984408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:34.985011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:34.985296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:34.985565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.012951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.013479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.013753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.014026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.014671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.016148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.045786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 37/80] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:34 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:35.046843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.063166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.065355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.105373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.161926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536bfdc-86de-4b9b-a1fe-7816dcefe524 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.199819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 44/81] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:35.199819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.200281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.200998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.201302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.201580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.202751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.204251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.206093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 38/82] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:35.207124 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.226116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.331204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d7003-9b04-4edb-b536-3d3a18967912 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.341510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de7bd0-33a8-4f3f-99c8-16aeb58f157e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.369812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.375281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.375932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.376607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.376957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.391631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.392754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.393039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.393316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.396000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.396853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:35.398195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.400093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 39/83] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:35.401042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:35.401818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:35.402622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:35.403394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 45/84] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:35.404033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.537967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f68223-7623-4033-8091-4de8a966e6f9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.538294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa212a1d-4e33-4ac2-b0b2-71229c478c31 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.547468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.578314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.578939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.579256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.579534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.587322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.588755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.589251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.589718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.591581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.593061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.594770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 40/85] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:35.596108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.657549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.670991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.672678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.704398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce50de-75ec-4515-a36b-e0d4b4f158c1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.740637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.741290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.741560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.741844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.760705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.783674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 46/86] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:35.784372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.816624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.818213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.893272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d4b88-3516-44ab-9020-883e5a6a1b18 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:35.903577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:35.931333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:35.932005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:35.932290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:35.932565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:35.933243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:35.933547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:35.934589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 41/87] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:35.934943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:35.935618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.945034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 47/88] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:35.945868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:35.987574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:36.040626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fa135-c6f6-4b41-a890-c8fa3f94b704 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.055701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf598be6-b524-42d0-abf0-51cbdd8e4478 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.076382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.077053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.077337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.077618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.078392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=6278e07627da40afa46d323a5ce985e4) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:36.078709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.080231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.088204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 42/89] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/role_assignments?user.id=6278e07627da40afa46d323a5ce985e4&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:36.089072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:36.092239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.092985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.093273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.093550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.094793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.096413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.098189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 48/90] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:35 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:36.098996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:36.144314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:36.208859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8ca6c-54cc-4ce3-a0ed-19dcc1d2a555 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.232042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:36.266681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164eb748-8a67-43d7-b07b-1f9c212ec169 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.287623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.288586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.288869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.289144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.290560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.292073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.293794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 43/91] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:36 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:36.295243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:36.334697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.335442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.335725 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.335999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.338667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.341520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.343235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 49/92] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:36 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:36.347671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:36.442904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0b08c-a6d1-4e07-b00c-ba5791da0603 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.488006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.488623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.488904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.489178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.489909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] RBAC: Adding query filter params (role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=6278e07627da40afa46d323a5ce985e4) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:36.490222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.491715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.509303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 44/93] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:36 2022] GET /identity/v3/role_assignments?user.id=6278e07627da40afa46d323a5ce985e4&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=5a02bc464efd499bb3230455492613ae => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:36.511071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:36.547468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:36.566952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4545d-5196-4d95-ac39-d55b88002c6e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.621103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:36.653032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.653656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.653937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.654207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.654951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=a4f3ff67989b433e88e7bd1a0058bb7d) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:36.655286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.660673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.675919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 50/94] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:36 2022] GET /identity/v3/role_assignments?user.id=a4f3ff67989b433e88e7bd1a0058bb7d&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:36.712632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ab6b8-e546-4ba2-8f49-e8ce3d234f0c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:36.751392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:36.752243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:36.752402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:36.752579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:36.833333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:36.834852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:36.899580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:36.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 45/95] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:36.960348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:37.079152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac739d-bddd-445e-8213-7f6afe5ec1c5 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:37.128724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a2184-dd74-4db6-82c4-0e02c0fe4ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:37.129519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a2184-dd74-4db6-82c4-0e02c0fe4ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:37.129811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a2184-dd74-4db6-82c4-0e02c0fe4ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:37.130090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a2184-dd74-4db6-82c4-0e02c0fe4ba6 None admin] PATH_INFO: `/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:37.131275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a2184-dd74-4db6-82c4-0e02c0fe4ba6 None admin] RBAC: Authorizing `identity:get_project(project_id=59a6cb44f24044908a99bcd67c1f4dd5)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:37.132734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a2184-dd74-4db6-82c4-0e02c0fe4ba6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:37.134539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 51/96] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:36 2022] GET /identity/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:37.140448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:37.259361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dce22c-7ae6-4d26-87b5-c545fb8cb7c9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:37.304020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:37.304635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:37.304915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:37.305192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:37.305923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=59a6cb44f24044908a99bcd67c1f4dd5, user.id=6278e07627da40afa46d323a5ce985e4) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:37.306231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:37.307785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5f6a3b-a56e-458d-a33e-8c47c964d532 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:37.314349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 46/97] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:37 2022] GET /identity/v3/role_assignments?user.id=6278e07627da40afa46d323a5ce985e4&scope.project.id=59a6cb44f24044908a99bcd67c1f4dd5&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:37.524799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98cf85-9722-4253-969e-8f8f62f5bcda None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:37.524981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98cf85-9722-4253-969e-8f8f62f5bcda None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:37.525274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98cf85-9722-4253-969e-8f8f62f5bcda None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:37.526939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 52/98] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:37.532150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:37.532437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:37.532712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:37.564381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:37.565607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:37.571931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:37.581155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd0764-e42e-4302-a1ce-7d2f4f6afa17 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:37.581464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd0764-e42e-4302-a1ce-7d2f4f6afa17 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:37.581715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd0764-e42e-4302-a1ce-7d2f4f6afa17 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:37.583102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 53/99] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:37.588589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:37.588877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:37.589158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:37.620212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:37.621430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:37.675619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:37.748995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 47/100] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:37.796615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:37.798681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 54/101] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:37.850025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:38.121206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd803b-14dc-4045-a68a-5307824a6210 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:38.162531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9ea2-add6-4d86-b541-3f7f47e22cf9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:38.187844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:38.188500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:38.188783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:38.189061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:38.195812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:38.197297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:38.198925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 55/102] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:38.208083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:38.234518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:38.235193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:38.235480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:38.235755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:38.252884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:38.258229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:38.261894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:38.264013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:38.264933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:38.265671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 48/103] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:38.272065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:38.394072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb259dff-983f-4945-9708-f76164b5ed06 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:38.441926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:38.442912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:38.443444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:38.443910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:38.444766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:38.446452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:38.497670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 56/104] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:17:38 2022] POST /identity/v3/projects => generated 302 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:38.532077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bac12-2175-427f-883c-beb11ce79c24 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:38.593782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:38.594419 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:38.594711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:38.594989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:38.595739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:38.596049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:38.597593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:38.612113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 49/105] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:38.617762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:38.760385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:38.806337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a96a8-7409-4cdf-a535-30df68223d57 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:38.859503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:38.859503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:38.859503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:38.859816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] PATH_INFO: `/v3/users/71c30ef7ec6b4174acf4f25c081b8514` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:38.885905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] RBAC: Authorizing `identity:get_user(user_id=71c30ef7ec6b4174acf4f25c081b8514)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:38.887469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:38.889388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 57/106] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:38 2022] GET /identity/v3/users/71c30ef7ec6b4174acf4f25c081b8514 => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:38.895111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:38.983593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:39.039505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7c6eb-2047-4b67-9196-314ec022d582 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:39.090942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd12b67-0c01-4f01-8669-9e3e3aa51a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:39.091973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd12b67-0c01-4f01-8669-9e3e3aa51a0e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.092537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd12b67-0c01-4f01-8669-9e3e3aa51a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.093040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd12b67-0c01-4f01-8669-9e3e3aa51a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.189947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd12b67-0c01-4f01-8669-9e3e3aa51a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:39.192081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd12b67-0c01-4f01-8669-9e3e3aa51a0e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:39.273351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:39.325243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 50/107] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:39.331489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:39.379631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2ab9f-7ac1-4a3b-903e-4f76f5b67032 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.379947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2ab9f-7ac1-4a3b-903e-4f76f5b67032 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.380186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2ab9f-7ac1-4a3b-903e-4f76f5b67032 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.381529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 51/108] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:39.388569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.388858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.389136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.418458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:39.419806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:39.525796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 52/109] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:39.554888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694d014-d7be-45ab-8ee8-ae43f4190a59 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:39.562815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:39.611874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:39.630604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:39.631301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.631589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.631865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.633029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:39.634486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:39.636472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 58/110] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:39 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:39.642237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:39.707598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:39.725245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925e333-3ef4-47b0-8653-dc95c50c7274 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:39.778421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:39.779078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.779381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.779653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.795231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e905a9-2140-4f99-b0aa-55484b1b428b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:39.797144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:39.802567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:39.803511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:39.804329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:39.816205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 53/111] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:39.821990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:39.862004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:39.862004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.862235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.862514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.863501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] RBAC: Adding query filter params (role.id=ade6fa30e2cc403b821cd127af76ebd7, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=71c30ef7ec6b4174acf4f25c081b8514) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:39.863820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:39.870508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f5341-42c9-446e-ab69-f091b4d48b61 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:39.879510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 59/112] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:39 2022] GET /identity/v3/role_assignments?user.id=71c30ef7ec6b4174acf4f25c081b8514&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=ade6fa30e2cc403b821cd127af76ebd7 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:39.940850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229471e-2353-40a3-a273-008fad5a300d None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.941434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229471e-2353-40a3-a273-008fad5a300d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.941909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229471e-2353-40a3-a273-008fad5a300d None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.943619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 60/113] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:39.950436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb0a7c-5095-45c7-adfe-552329a577c6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:39.950837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb0a7c-5095-45c7-adfe-552329a577c6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:39.951164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb0a7c-5095-45c7-adfe-552329a577c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:39.985173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-52eb0a7c-5095-45c7-adfe-552329a577c6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:39.986991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-52eb0a7c-5095-45c7-adfe-552329a577c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:40.007142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35204612-be93-42d4-90ad-3cdddb865d8f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:40.107341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:40.107969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.108247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.108519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.109210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:40.109520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:40.111545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:40.132313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 54/114] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:40.137689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:40.139924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:40.166514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 61/115] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:40.166829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:40.167761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7032a-1cd7-4286-9d79-2081d959a6a1 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.168165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7032a-1cd7-4286-9d79-2081d959a6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.168417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7032a-1cd7-4286-9d79-2081d959a6a1 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.172162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 62/116] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:40.181735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.182018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.182295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.225508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:40.226698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:40.302967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b01189-875a-42d4-9c2e-7a1d353f7433 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:40.321761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 63/117] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:40.322535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:40.342758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:40.343485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.343777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.344162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.345594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:40.347447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:40.349454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 55/118] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:40 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:40.355128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:40.491591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:40.524078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb815c47-f77f-423d-ade3-01b020f291a7 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:40.542613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de884037-ad2d-4b89-890a-5f9ac2a9c9e2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:40.574979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:40.599978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e2261-87d4-42a9-aaec-c868d3009e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:40.600629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2261-87d4-42a9-aaec-c868d3009e17 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.600913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2261-87d4-42a9-aaec-c868d3009e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.601192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e2261-87d4-42a9-aaec-c868d3009e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.615430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:40.616074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.616366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.616647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.620036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:40.621516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:40.624134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 56/119] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:40 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:40.624472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e2261-87d4-42a9-aaec-c868d3009e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:40.625378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c79e2261-87d4-42a9-aaec-c868d3009e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:40.634854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:40.635804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:40.636534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:40.637092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 64/120] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:40.638266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f4b3c-1746-46eb-9b7d-33686e1e0675 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.638746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f4b3c-1746-46eb-9b7d-33686e1e0675 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.639036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f4b3c-1746-46eb-9b7d-33686e1e0675 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.640637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 65/121] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:40.645465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f4b3c-1746-46eb-9b7d-33686e1e0675 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:40.753819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579716e-8ee2-4582-8e0e-eab942659b80 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:40.786770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f4b3c-1746-46eb-9b7d-33686e1e0675 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:40.791126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:40.792143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.792734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.793260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.805077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:40.810716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:40.811860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:40.812614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:40.815791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 57/122] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:17:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:40.816780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:40.830346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:40.836922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ee6e0-e6ce-4ba7-bff1-de43ea7eea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:40.837600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee6e0-e6ce-4ba7-bff1-de43ea7eea3d None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.837884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee6e0-e6ce-4ba7-bff1-de43ea7eea3d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.838181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ee6e0-e6ce-4ba7-bff1-de43ea7eea3d None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/6278e07627da40afa46d323a5ce985e4/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.841167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ee6e0-e6ce-4ba7-bff1-de43ea7eea3d None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c2beee9242a98bbbec05ac9e6717, user_id=6278e07627da40afa46d323a5ce985e4, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:40.843063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ee6e0-e6ce-4ba7-bff1-de43ea7eea3d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:40.868854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 66/123] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:40 2022] PUT /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/6278e07627da40afa46d323a5ce985e4/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:17:40.869874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:40.870170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:40.871461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:40.910500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:40.911795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:40.988180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.013527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202b857-aa67-4a43-ba04-33c9643fc2d4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.042111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 67/124] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:41.043013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.093454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.094081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.094418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.094703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.095446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:41.095804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.097310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.110316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 58/125] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.112408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.181762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0b335-17d6-44c7-9ce2-2f0217ab2550 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.191997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.231765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.232898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.233379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.233788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.234524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:41.235127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.237070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.247281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 68/126] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.248591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.251162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091591db-a2f5-49e6-a0f8-03b261a656e6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.290591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.321315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.321959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.322260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.322514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.341583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:41.349652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:41.350510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:41.351309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 59/127] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:41.352650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.421764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1927799-e058-45bc-a63e-a13acf9d38c7 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.488646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.489294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.489576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.489855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.490999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.492538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.495385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 69/128] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.496169 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.549737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c2c00-525f-4d6d-95b3-0e35e8efde11 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.584033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.592121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.592767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.593054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.593336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.595662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.597605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.600199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 60/129] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.600986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.650450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e061718-1224-4123-85e0-1ca2a8f5bc80 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.652562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.700807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.701788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.702296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.703252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.704131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:41.704698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.706944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8336a31-64e4-492c-89de-4ac7117d2a93 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.717630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 70/130] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.719825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce356496-745d-41e2-a01f-2d1e9793f0c9 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.720925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce356496-745d-41e2-a01f-2d1e9793f0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.721378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce356496-745d-41e2-a01f-2d1e9793f0c9 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.727509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 71/131] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:41.728610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce356496-745d-41e2-a01f-2d1e9793f0c9 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.739809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9726064-40a7-4e7c-a504-917ae57877dd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.783900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.784850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.785361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.785805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.787379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.789151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.791199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 61/132] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.792382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:41.867502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce356496-745d-41e2-a01f-2d1e9793f0c9 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.885854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.911771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d9a43-62fe-4b3b-9006-fe5e1f53ccaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.912533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d9a43-62fe-4b3b-9006-fe5e1f53ccaf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.912943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d9a43-62fe-4b3b-9006-fe5e1f53ccaf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.913210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d9a43-62fe-4b3b-9006-fe5e1f53ccaf None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.914514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d9a43-62fe-4b3b-9006-fe5e1f53ccaf None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.916189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d9a43-62fe-4b3b-9006-fe5e1f53ccaf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.919131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 72/133] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.919131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.919433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.919678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.938419 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd504-d68f-41fe-98e4-f052f56fae6d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:41.939615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:41.941164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:41.970209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:41.985320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:41.986036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:41.986324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:41.986605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:41.987794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:41.989865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:41.992278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 62/134] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:41 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:41.993043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.019871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 73/135] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:42.027803 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.123129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab6d80-68d8-484d-b070-57a2244a091a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.153585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d3a5d-6da2-46cf-bf45-38e2da7a480d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.164428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.165103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.165392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.165675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/a4f3ff67989b433e88e7bd1a0058bb7d/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.167642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c2beee9242a98bbbec05ac9e6717, user_id=a4f3ff67989b433e88e7bd1a0058bb7d, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:42.169110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:42.171417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:42.186917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 63/136] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:41 2022] PUT /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/a4f3ff67989b433e88e7bd1a0058bb7d/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:17:42.187570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.196998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.198158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.198684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.199222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/6278e07627da40afa46d323a5ce985e4/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.202044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c2beee9242a98bbbec05ac9e6717, user_id=6278e07627da40afa46d323a5ce985e4, role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:42.204009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:42.235482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 74/137] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:42 2022] PUT /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/6278e07627da40afa46d323a5ce985e4/roles/5a02bc464efd499bb3230455492613ae => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:17:42.236329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.319493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:42.397785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5491-3f60-499f-8950-e1f6591bf997 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.434551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831346a-7952-40fe-897c-7bb0c0c59838 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.438357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb3937-a9e5-44ce-b221-3fa255cd9d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.438502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb3937-a9e5-44ce-b221-3fa255cd9d1e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.438731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb3937-a9e5-44ce-b221-3fa255cd9d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.439580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb3937-a9e5-44ce-b221-3fa255cd9d1e None admin] PATH_INFO: `/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.440817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb3937-a9e5-44ce-b221-3fa255cd9d1e None admin] RBAC: Authorizing `identity:get_project(project_id=59a6cb44f24044908a99bcd67c1f4dd5)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:42.442276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb3937-a9e5-44ce-b221-3fa255cd9d1e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:42.443959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 64/138] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:42 2022] GET /identity/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5 => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:42.446331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b0fda-24c7-4b85-808e-8fba3b641870 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.446618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b0fda-24c7-4b85-808e-8fba3b641870 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.446895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b0fda-24c7-4b85-808e-8fba3b641870 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.450391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 65/139] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:42.456387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b0fda-24c7-4b85-808e-8fba3b641870 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.478345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef5bd5-0443-4d33-808d-7fc0f9ce22ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.478994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef5bd5-0443-4d33-808d-7fc0f9ce22ce None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.479321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef5bd5-0443-4d33-808d-7fc0f9ce22ce None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.479605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef5bd5-0443-4d33-808d-7fc0f9ce22ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.480749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef5bd5-0443-4d33-808d-7fc0f9ce22ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:42.482332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef5bd5-0443-4d33-808d-7fc0f9ce22ce None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:42.487387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 75/140] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:42.487387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.487387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.487387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.504842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:42.506051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:42.558995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:42.581140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 76/141] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:42.582198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.603972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:42.639840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b0fda-24c7-4b85-808e-8fba3b641870 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.691840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058a8ec-5936-42eb-a6be-3c2020eee1f0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.728345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.729120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.729333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.729463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.729670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] PATH_INFO: `/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5/users/6278e07627da40afa46d323a5ce985e4/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.729931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.730212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.730590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:42.731292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:42.732766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:42.737037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] RBAC: Authorizing `identity:create_grant(project_id=59a6cb44f24044908a99bcd67c1f4dd5, user_id=6278e07627da40afa46d323a5ce985e4, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:42.738504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:42.759328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 77/142] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:42 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:42.760342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:42.773658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 66/143] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:42 2022] PUT /identity/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5/users/6278e07627da40afa46d323a5ce985e4/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:17:42.871599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:42.919301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9deae5d-206e-4e45-8acd-8725e46e4799 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:42.975405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:42.976291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:42.976719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:42.977137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.002258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:43.007817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:43.008854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:43.009801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:43.015990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 78/144] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:43.023068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:43.165588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92b41d-7df9-4a73-9d8c-8ed8652fae7c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:43.197599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:43.213079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:43.213702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.213982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.214253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.214919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:43.215255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:43.216651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:43.231174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 67/145] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:43.236775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:43.371887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a810f-87d5-4674-b792-627d631ce707 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:43.415066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2e5f3-53df-420f-a264-092ec64f99f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:43.416172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2e5f3-53df-420f-a264-092ec64f99f2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.416652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2e5f3-53df-420f-a264-092ec64f99f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.417129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2e5f3-53df-420f-a264-092ec64f99f2 None admin] PATH_INFO: `/v3/users/71c30ef7ec6b4174acf4f25c081b8514` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.424470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2e5f3-53df-420f-a264-092ec64f99f2 None admin] RBAC: Authorizing `identity:get_user(user_id=71c30ef7ec6b4174acf4f25c081b8514)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:43.426231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2e5f3-53df-420f-a264-092ec64f99f2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:43.428115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 79/146] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:43 2022] GET /identity/v3/users/71c30ef7ec6b4174acf4f25c081b8514 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:43.436203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:43.510588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3333aef-c2cc-4c95-99af-a4b10e6d76f5 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.511231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3333aef-c2cc-4c95-99af-a4b10e6d76f5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.511741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3333aef-c2cc-4c95-99af-a4b10e6d76f5 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.513491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 80/147] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:43.523686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.524286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.524776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.541849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:43.560192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:43.561812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:43.586376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a437a30b-02f6-4d95-bc38-4250072cf928 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:43.637277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:43.638259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.639338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.639825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.641571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:43.643466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:43.645572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 68/148] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:43 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:43.651607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:43.732822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:43.744399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 81/149] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:43.784424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec9d59d-c8f6-415d-a1ed-e42c08718bf2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:43.799832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:43.839204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:43.840893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.842357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.842946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/71c30ef7ec6b4174acf4f25c081b8514/roles/ade6fa30e2cc403b821cd127af76ebd7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.847172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c2beee9242a98bbbec05ac9e6717, user_id=71c30ef7ec6b4174acf4f25c081b8514, role_id=ade6fa30e2cc403b821cd127af76ebd7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:43.850636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:43.876115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 69/150] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:43 2022] PUT /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717/users/71c30ef7ec6b4174acf4f25c081b8514/roles/ade6fa30e2cc403b821cd127af76ebd7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:17:43.955676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02200fa7-beae-47f5-849f-1829a303fcdd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:43.993095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:43.993747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:43.994024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:43.994293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:43.996334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:44.007615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:44.012650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:44.013599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:44.014361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 82/151] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:44.020138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:44.139691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321257b-1265-4989-a55f-649bae832c87 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:44.164946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:44.189716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:44.190357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:44.190609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:44.190882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:44.191608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:44.191917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:44.193326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:44.205363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 70/152] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:44.211357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:44.331868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4f48e-6357-435b-963a-2e4f735ca034 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:44.369037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:44.369734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:44.370014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:44.370288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:44.371573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:44.373051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:44.375056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 83/153] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:44 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:44.381240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:44.496031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f6c8c-ebfe-4db6-9601-214525c6c646 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:44.542516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:44.543156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:44.543442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:44.543711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:44.592139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:44.620136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:44.621686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:44.733395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 71/154] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:44.739859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:44.828403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:44.877698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13361bd4-0c2c-42a9-a203-c79eae832c2c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:44.920372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467eade-6331-4340-9ac2-a498d2d0042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:44.921011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467eade-6331-4340-9ac2-a498d2d0042b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:44.921289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467eade-6331-4340-9ac2-a498d2d0042b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:44.921562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467eade-6331-4340-9ac2-a498d2d0042b None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:44.922697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467eade-6331-4340-9ac2-a498d2d0042b None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:44.924235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c467eade-6331-4340-9ac2-a498d2d0042b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:44.925879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 84/155] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:44 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:44.931919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:44.996871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720d507-9d03-4325-82a3-040b6bd91dff None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:44.997189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720d507-9d03-4325-82a3-040b6bd91dff None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:44.997419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720d507-9d03-4325-82a3-040b6bd91dff None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:44.998835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 85/156] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:45.031204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:45.031842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:45.032320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:45.049470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:45.055712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:45.057223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:45.078666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd4a66-296f-40ef-8e23-74e193b27224 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:45.143625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:45.144705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:45.145181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:45.145657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:45.146614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=6278e07627da40afa46d323a5ce985e4) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:45.147308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:45.149027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:45.164208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 86/157] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:45.164915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 72/158] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:44 2022] GET /identity/v3/role_assignments?user.id=6278e07627da40afa46d323a5ce985e4&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:45.208433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:45.282950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:45.339158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b9ce1c-7e9d-459f-9df9-6fbb13627d21 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:45.374893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:45.375909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:45.376867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:45.377286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:45.389321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:45.395071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:45.396280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:45.397326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:45.399705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 87/159] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:45.412238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:45.581059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050337f-56b0-4165-957b-9aa842566a87 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:45.625737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:45.626820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:45.627114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:45.627399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:45.628072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:45.628385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:45.629782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:45.639396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:45.643671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 73/160] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:45.649011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:45.820902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f545b-c86c-4512-854d-91df061038ff None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:45.863686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:45.891938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055baa-9d67-476d-90d3-38a033ae022d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:45.892606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055baa-9d67-476d-90d3-38a033ae022d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:45.892891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055baa-9d67-476d-90d3-38a033ae022d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:45.893210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055baa-9d67-476d-90d3-38a033ae022d None admin] PATH_INFO: `/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:45.900004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055baa-9d67-476d-90d3-38a033ae022d None admin] RBAC: Authorizing `identity:get_user(user_id=a4f3ff67989b433e88e7bd1a0058bb7d)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:45.901526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055baa-9d67-476d-90d3-38a033ae022d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:45.903423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 88/161] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:45 2022] GET /identity/v3/users/a4f3ff67989b433e88e7bd1a0058bb7d => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:45.909283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:46.066204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11697923-6874-4817-ba3a-6c3477d20544 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:46.113651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:46.114276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.114557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.114859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.206362 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:46.207819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:46.211428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:46.237224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a64bc1-540f-49ee-9f24-c4dd368f96b5 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.237493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a64bc1-540f-49ee-9f24-c4dd368f96b5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.237766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a64bc1-540f-49ee-9f24-c4dd368f96b5 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.239175 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 89/162] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:46.248503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.248790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.249085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.300443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:46.301681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:46.353998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 74/163] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:46.359725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:46.441206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 90/164] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:46.500212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a1198-09ea-40c7-9ac9-d6e5ee46ae51 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:46.501799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:46.504191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:46.545278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:46.545924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.546215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.546493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.547669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:46.549193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:46.550850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 75/165] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:46 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:46.557039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:46.679305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edb0de-1b76-4233-9770-eb64075f6d3d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:46.703194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:46.724380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c78bf-37d3-46c3-b0ee-82a27d91b499 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:46.755946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729a4a0-1db4-4a92-938c-d7324f3e07cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:46.756526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729a4a0-1db4-4a92-938c-d7324f3e07cf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.756951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729a4a0-1db4-4a92-938c-d7324f3e07cf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.757231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729a4a0-1db4-4a92-938c-d7324f3e07cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.769670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:46.770289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.770569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.770843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.771702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=a4f3ff67989b433e88e7bd1a0058bb7d) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:46.772017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:46.773469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:46.781128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729a4a0-1db4-4a92-938c-d7324f3e07cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:46.785338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 76/166] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:46 2022] GET /identity/v3/role_assignments?user.id=a4f3ff67989b433e88e7bd1a0058bb7d&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3729a4a0-1db4-4a92-938c-d7324f3e07cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:46.786049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:46.788258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:46.790464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:46.795058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:46.795305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 91/167] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:17:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:46.845810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:46.969016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4881e9b-9481-4cd4-bae0-60298b6673d5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:46.997165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61a185-a929-4955-86d6-367abed59fcf None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:46.997809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61a185-a929-4955-86d6-367abed59fcf None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:46.997809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61a185-a929-4955-86d6-367abed59fcf None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:46.999084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 92/168] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:47.013468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.013773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.026646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:47.027289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.027570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.027842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.028516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:47.028817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:47.030429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:47.046375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 77/169] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:17:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:47.052028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:47.052238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:47.053399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:47.172453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 93/170] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:47.212883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:47.247515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:47.257680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f5539-f262-4930-9770-875df6f35ac3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:47.335348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:47.335995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.336275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.336554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.337723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:47.339210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:47.340811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 78/171] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:47 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:47.363858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:47.387739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664b92-4860-4777-8d23-4355f1644b18 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:47.397365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:47.433116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5e013-627e-48b9-9e60-fa0029052801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:47.433772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e013-627e-48b9-9e60-fa0029052801 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.434058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e013-627e-48b9-9e60-fa0029052801 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.434333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e013-627e-48b9-9e60-fa0029052801 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.447049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc5e013-627e-48b9-9e60-fa0029052801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-fbc5e013-627e-48b9-9e60-fa0029052801 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:47.453079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:47.455364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 94/172] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:47.457303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0a152-5fda-4c73-8a91-7af402aa5a26 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.459023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0a152-5fda-4c73-8a91-7af402aa5a26 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.459023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0a152-5fda-4c73-8a91-7af402aa5a26 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.459023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 95/173] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:47.462316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0a152-5fda-4c73-8a91-7af402aa5a26 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:47.549958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd5d23-d448-45d3-966a-19b81bef6a78 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:47.605388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d093e-e5e7-41c2-8193-efb12dbac0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:47.606360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d093e-e5e7-41c2-8193-efb12dbac0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.606841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d093e-e5e7-41c2-8193-efb12dbac0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.607363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d093e-e5e7-41c2-8193-efb12dbac0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.659962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0a152-5fda-4c73-8a91-7af402aa5a26 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:47.660415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:47.707507 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:47.708791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.708972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d093e-e5e7-41c2-8193-efb12dbac0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:47.709477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.710011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.710461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d093e-e5e7-41c2-8193-efb12dbac0b6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:47.711126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:47.711678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:47.714537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f2dab-bf3f-4fa7-a8f5-436f97cfb306 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:47.726560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 96/174] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:47.728875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.729387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.730131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.753862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:47.755421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:47.788370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:47.840005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 79/175] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:17:47.841782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e9148-84ef-4f97-b604-58c92551bacb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:47.842348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e9148-84ef-4f97-b604-58c92551bacb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:47.842859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e9148-84ef-4f97-b604-58c92551bacb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:47.844551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 80/176] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:47.846177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e9148-84ef-4f97-b604-58c92551bacb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:47.850650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 97/177] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:47.850650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:47.923234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:47.967764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6548a34-e3ee-4107-aa7c-b02c1cb1a599 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:47.970790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e9148-84ef-4f97-b604-58c92551bacb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.007478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4319e75-83e1-4223-875e-f20e20d5585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.008265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4319e75-83e1-4223-875e-f20e20d5585d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.008436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4319e75-83e1-4223-875e-f20e20d5585d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.008786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4319e75-83e1-4223-875e-f20e20d5585d None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.009482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.009839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4319e75-83e1-4223-875e-f20e20d5585d None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.010113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.010417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.010662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] PATH_INFO: `/v3/users/6278e07627da40afa46d323a5ce985e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.011331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4319e75-83e1-4223-875e-f20e20d5585d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.011830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] RBAC: Authorizing `identity:get_user(user_id=6278e07627da40afa46d323a5ce985e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.013204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 98/178] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:47 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:48.013355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.019377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 81/179] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:47 2022] GET /identity/v3/users/6278e07627da40afa46d323a5ce985e4 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:48.019377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.019767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3082e-6950-492e-bb08-8d267238eaee None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.020020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3082e-6950-492e-bb08-8d267238eaee None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.020293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3082e-6950-492e-bb08-8d267238eaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.039072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-72f3082e-6950-492e-bb08-8d267238eaee None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:48.040313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-72f3082e-6950-492e-bb08-8d267238eaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:48.064958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.121184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 99/180] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:48.121184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e803aa-47ff-4692-afdf-9e74e69d24fb None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.121184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e803aa-47ff-4692-afdf-9e74e69d24fb None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.121184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e803aa-47ff-4692-afdf-9e74e69d24fb None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.121184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 100/181] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:48.121184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e803aa-47ff-4692-afdf-9e74e69d24fb None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.129690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2e590-7ad1-45e3-bc04-3d20260e6af3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.159885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.168632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.169517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.169973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.170422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.181952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:48.187385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:48.188232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:48.189083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:48.190974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 82/182] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:48.192549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.236058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e803aa-47ff-4692-afdf-9e74e69d24fb None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.269994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.270586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.270862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.271156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.271882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] RBAC: Adding query filter params (role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=6278e07627da40afa46d323a5ce985e4) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:48.272182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.273622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5420dc0-7145-4b74-af78-8316bae12d7c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.279257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 101/183] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/role_assignments?user.id=6278e07627da40afa46d323a5ce985e4&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=5a02bc464efd499bb3230455492613ae => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:48.280460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.280740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.281009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.300582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ba91c-6259-457f-a73a-8d4ede14d9c3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.310833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:48.312342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:48.333538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.343505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.344116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.344398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.344671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.397869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 102/184] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:48.398464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.425666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.439955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.443019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.525005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae555b-28b4-4e6a-87ec-55001220c825 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.552937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 83/185] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:48.553780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.559328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd578e7e-5da5-45e1-be69-7f2c4d94ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.559968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd578e7e-5da5-45e1-be69-7f2c4d94ed59 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.560271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd578e7e-5da5-45e1-be69-7f2c4d94ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.560540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd578e7e-5da5-45e1-be69-7f2c4d94ed59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.572255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd578e7e-5da5-45e1-be69-7f2c4d94ed59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-fd578e7e-5da5-45e1-be69-7f2c4d94ed59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:48.576914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:48.577768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 103/186] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:48.584763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765ab6a-2f59-4c5b-81af-139da8574563 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.585609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765ab6a-2f59-4c5b-81af-139da8574563 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.585609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765ab6a-2f59-4c5b-81af-139da8574563 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.585609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 104/187] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:48.585609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765ab6a-2f59-4c5b-81af-139da8574563 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.597720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.647195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.656895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57059db7-a2c9-4a9f-b244-736ec36ca832 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.684448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765ab6a-2f59-4c5b-81af-139da8574563 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.690980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.691616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.691890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.692160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.692849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:48.693149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.694529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.702738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 84/188] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:48.703576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.721721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.722414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.722700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.722981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.733274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:48.738972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:48.740100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 105/189] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:48.744056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.811961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.813209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e0b81-da35-4440-af90-44b71af8f4a8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.843331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779992c-f740-407a-a609-620866e9e9b8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:48.848075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da83a2-80ee-4d06-b7e2-b3c002b79736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.848710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da83a2-80ee-4d06-b7e2-b3c002b79736 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.848993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da83a2-80ee-4d06-b7e2-b3c002b79736 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.849268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da83a2-80ee-4d06-b7e2-b3c002b79736 None admin] PATH_INFO: `/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.850384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da83a2-80ee-4d06-b7e2-b3c002b79736 None admin] RBAC: Authorizing `identity:get_project(project_id=59a6cb44f24044908a99bcd67c1f4dd5)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.851885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da83a2-80ee-4d06-b7e2-b3c002b79736 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.853529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 85/190] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/projects/59a6cb44f24044908a99bcd67c1f4dd5 => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:48.854629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.854917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.855211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.872982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:48.874187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:48.882567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:48.883207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:48.883554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:48.883826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:48.884506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:48.884809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:48.886212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:48.889061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:48.894894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 106/191] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:48.895738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.943973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 86/192] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:48.948439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:48.995906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a23bdd-6f5f-442b-a65a-7771d95a5255 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.029755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.030756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.031043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.031353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] PATH_INFO: `/v3/users/aa68451f072d4eeebdb68b90cad86309` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.033269 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.046280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] RBAC: Authorizing `identity:get_user(user_id=aa68451f072d4eeebdb68b90cad86309)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.047781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.048024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaddfd9-ee93-4265-9e1f-82a6d4aa3506 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.049494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 107/193] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/users/aa68451f072d4eeebdb68b90cad86309 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:49.051894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.083556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.084190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.084464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.084731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.085409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:49.085709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.087783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.095019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 87/194] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:49.095826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.109803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.152485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84df8-280e-4365-b356-845b76015d1a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.185942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.186792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.187067 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.187368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.188086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=59a6cb44f24044908a99bcd67c1f4dd5, user.id=6278e07627da40afa46d323a5ce985e4) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:49.188397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.190327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.195142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc5d50-5fc1-41c7-9967-5b78470aa55d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.195932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 108/195] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/role_assignments?user.id=6278e07627da40afa46d323a5ce985e4&scope.project.id=59a6cb44f24044908a99bcd67c1f4dd5&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:49.196978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.234575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.234803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.234940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.235203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] PATH_INFO: `/v3/users/aa68451f072d4eeebdb68b90cad86309` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.236928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] RBAC: Authorizing `identity:get_user(user_id=aa68451f072d4eeebdb68b90cad86309)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.238398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.244755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 88/196] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/users/aa68451f072d4eeebdb68b90cad86309 => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:49.244755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.265027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.318176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401aa482-bac5-4d83-9f31-f752d3fce48b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.336581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.356650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75756fa0-133f-4844-8999-de983235f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.357660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756fa0-133f-4844-8999-de983235f70a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.358813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22930ce-4bbb-4c86-a4ee-620fc03afe71 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.358925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756fa0-133f-4844-8999-de983235f70a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.358925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75756fa0-133f-4844-8999-de983235f70a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.359724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75756fa0-133f-4844-8999-de983235f70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.361793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75756fa0-133f-4844-8999-de983235f70a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.366012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 109/197] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:49.366012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75756fa0-133f-4844-8999-de983235f70a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.398844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.399733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.400016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.400290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.473721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75756fa0-133f-4844-8999-de983235f70a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.474949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.476489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.497460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.509476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.510106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.510383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.510656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] PATH_INFO: `/v3/users/71c30ef7ec6b4174acf4f25c081b8514` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.511833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] RBAC: Authorizing `identity:get_user(user_id=71c30ef7ec6b4174acf4f25c081b8514)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.513292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.514875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 110/198] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/users/71c30ef7ec6b4174acf4f25c081b8514 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:49.519589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.556678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.578152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 89/199] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:49.578713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.618404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0842ca8-26c2-4e25-a0ac-f04c4ecf2286 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.652391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.653000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.653279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.653555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.677086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97834-1da4-4485-9b63-41692132d67c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.711267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.712055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.712657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.712971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.713196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.713842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.715320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.722153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:49.724188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:49.735056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 90/200] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:49 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:49.746108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.793028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.849124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 111/201] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:49.854844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:49.872085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3281f-1793-4549-91d1-fcd3b8a38e8f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:49.916503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:49.916817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:49.917192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:49.917543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:49.968069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:49.987359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce4c77-ea92-4a8e-b4ed-aedc0f780a14 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:50.005821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.008010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.030834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:50.031579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:50.031861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:50.032650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:50.033901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.035378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.037238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 112/202] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:50.038004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:50.057454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:50.150597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 91/203] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:50.152093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:50.162599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d038c-c89e-403a-9ee3-c5bbd6adfbae None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:50.200579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:50.201215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:50.201496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:50.201770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:50.202904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.205047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.211126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 113/204] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:50 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:50.211126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:50.242258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:50.287902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d9e88-d27d-4f73-ad77-a5a1b719e4cf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:50.323215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:50.323832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:50.324114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:50.324390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:50.325126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=aa68451f072d4eeebdb68b90cad86309) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:50.325439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.326883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.330710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:50.335902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 92/205] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:50 2022] GET /identity/v3/role_assignments?user.id=aa68451f072d4eeebdb68b90cad86309&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:50.336536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:50.338995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8c12f-1d78-4ec1-b990-1225de8f7650 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:50.385363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:50.386136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:50.386469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:50.386784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:50.388148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.390662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.393511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 114/206] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:50 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:50.399603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:50.525898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:50.536207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38684b4f-6270-4993-9445-4dadec171f35 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:50.558049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160c36a-131c-43f8-b3eb-c7ee3ba4a52e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:50.575533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:50.576148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:50.576422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:50.576690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:50.577485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] RBAC: Adding query filter params (role.id=ade6fa30e2cc403b821cd127af76ebd7, scope.project.id=e319c2beee9242a98bbbec05ac9e6717, user.id=71c30ef7ec6b4174acf4f25c081b8514) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:50.577791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.579899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d3a3e-3b79-4d36-a0f4-2a4592526b9f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.586599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 115/207] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:50 2022] GET /identity/v3/role_assignments?user.id=71c30ef7ec6b4174acf4f25c081b8514&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=ade6fa30e2cc403b821cd127af76ebd7 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:50.610620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:50.611281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:50.611561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:50.611830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:50.612625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] RBAC: Adding query filter params (role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=aa68451f072d4eeebdb68b90cad86309) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:50.612936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:50.614384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4b7f8-3887-4395-82aa-356727d3fcf9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:50.620964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 93/208] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:50 2022] GET /identity/v3/role_assignments?user.id=aa68451f072d4eeebdb68b90cad86309&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=5a02bc464efd499bb3230455492613ae => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:51.961442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acda3e-1d48-489d-9f63-7f7e7284765f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:51.961957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acda3e-1d48-489d-9f63-7f7e7284765f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:51.961957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acda3e-1d48-489d-9f63-7f7e7284765f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:51.963513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 116/209] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:51.969798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc248cc1-e719-4930-b76e-524e22bae277 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:51.970275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc248cc1-e719-4930-b76e-524e22bae277 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:51.970546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc248cc1-e719-4930-b76e-524e22bae277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:51.990196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bc248cc1-e719-4930-b76e-524e22bae277 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:51.991415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bc248cc1-e719-4930-b76e-524e22bae277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:52.056364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:52.069641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 94/210] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:52.088038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acda3e-1d48-489d-9f63-7f7e7284765f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.103580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f1d4-ffd4-42d4-a8d5-7425bd51de43 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.103944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f1d4-ffd4-42d4-a8d5-7425bd51de43 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.104549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3f1d4-ffd4-42d4-a8d5-7425bd51de43 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.105793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 95/211] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:52.111551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.112008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.112440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.130717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:52.132201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:52.211329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 96/212] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:52.215951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acda3e-1d48-489d-9f63-7f7e7284765f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.230477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.253561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.254201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.254476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.254744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.265984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:52.271812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:52.272667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:52.273441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 117/213] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:52.279152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.303573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:52.340942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa02b16b-cb30-4bed-b653-93d7f5f2a2b5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.367543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:52.379949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.380987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.381465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.381955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.383311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:52.385068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:52.386927 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 97/214] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:52.393103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.393564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d1a23-1e70-40ae-a951-0edf0ed5bea6 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.432301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.432919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.433194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.433464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.434128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:52.434427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:52.436101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:52.449855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 118/215] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:52.451154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.506904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1004b-4db8-46e5-bcae-b991364e3ef9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.543533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:52.558586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a2c06-92cf-4559-8415-b377f1c6d430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.560204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a2c06-92cf-4559-8415-b377f1c6d430 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.560483 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a2c06-92cf-4559-8415-b377f1c6d430 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.560802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a2c06-92cf-4559-8415-b377f1c6d430 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.561754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a2c06-92cf-4559-8415-b377f1c6d430 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:52.563262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a2c06-92cf-4559-8415-b377f1c6d430 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:52.565306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ec6e0-be2d-4ae3-b888-580b25312ee9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.585674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 98/216] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:52 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:52.586976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90742a-bce7-473f-8503-ee0856e5a8d9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.587296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90742a-bce7-473f-8503-ee0856e5a8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.587663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90742a-bce7-473f-8503-ee0856e5a8d9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.593225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 99/217] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:52.595315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.595600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.595943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.616774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.617431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.617712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.617982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] PATH_INFO: `/v3/users/ad9bb028917c4eaa84443f62df561c9b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.618561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:52.620016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:52.636456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9bb028917c4eaa84443f62df561c9b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:52.638679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:52.643297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 119/218] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/users/ad9bb028917c4eaa84443f62df561c9b => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:52.648197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.679890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:52.726644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 100/219] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:52.746894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:52.788490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8197cc0-1a38-4ad3-bde9-db887e1cb387 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.824399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c4cd-24ec-4a24-b617-4d5be2e4ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.825017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c4cd-24ec-4a24-b617-4d5be2e4ae7e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.825290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c4cd-24ec-4a24-b617-4d5be2e4ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.825559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c4cd-24ec-4a24-b617-4d5be2e4ae7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.837390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:52.883520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad987b6-486a-4617-93d7-b6af3cc6b07f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:52.909065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77c4cd-24ec-4a24-b617-4d5be2e4ae7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:52.910469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77c4cd-24ec-4a24-b617-4d5be2e4ae7e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:52.933639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c7c5f-0a75-4be2-b75b-bcdf19d21822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:52.934280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c7c5f-0a75-4be2-b75b-bcdf19d21822 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.934562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c7c5f-0a75-4be2-b75b-bcdf19d21822 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.934836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c7c5f-0a75-4be2-b75b-bcdf19d21822 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.946894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c7c5f-0a75-4be2-b75b-bcdf19d21822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-436c7c5f-0a75-4be2-b75b-bcdf19d21822 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:52.951742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:52.952674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:52.953899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 101/220] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:52.955302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32c66-90a2-4cb6-860f-370787a66316 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:52.955302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32c66-90a2-4cb6-860f-370787a66316 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:52.955302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc32c66-90a2-4cb6-860f-370787a66316 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:52.956432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 102/221] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:52.959300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc32c66-90a2-4cb6-860f-370787a66316 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.018329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.043202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 120/222] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:53.043202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.043202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.043202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.061439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:53.062789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:53.102388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.105017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc32c66-90a2-4cb6-860f-370787a66316 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.147158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.147970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.148302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.148688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.149458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:53.149884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.150908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 121/223] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:53.151537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.152397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.161365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 103/224] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:53.166367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.281128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af60d6-1a1f-4d64-ab31-b067ec301c49 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.294780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.299580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c0c43-c4eb-4b26-b328-7845f047cd44 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.320904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.321784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.322185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.322655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.323971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.325643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.327776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 122/225] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:53.329497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.343502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.344566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.345273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.345552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] PATH_INFO: `/v3/users/aa68451f072d4eeebdb68b90cad86309` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.346945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] RBAC: Authorizing `identity:get_user(user_id=aa68451f072d4eeebdb68b90cad86309)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.349227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.351846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 104/226] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/users/aa68451f072d4eeebdb68b90cad86309 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:53.352635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.376699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.448975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f85a92-6818-4882-8f8f-5d1eb67285ad None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.486827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.487778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.488070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.488448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.496505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50d14b-bdaf-43c6-9218-a55cbd8194cc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.501485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:53.506576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:53.507665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:53.508626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:53.515150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 123/227] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:53.515150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.548486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.549481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.549909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.550354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.551330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=ad9bb028917c4eaa84443f62df561c9b) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:53.551786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.553379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.560007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 105/228] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/role_assignments?user.id=ad9bb028917c4eaa84443f62df561c9b&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:53.561248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.598443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.621749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.627505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f9da7-efd9-40da-9162-5bad503f6912 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.666618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.667480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.667865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.668245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.670403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.672442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.675275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 124/229] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:53.680871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.689429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834c0b7-85a9-4bbb-be3f-d6b3b97d7e16 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.759811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.760672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.763799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.764250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.765127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:53.765708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.768907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.793150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 106/230] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:53.794431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:53.897297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a08bb-4201-48c3-93a5-04de7ebc6377 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.945649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:53.946335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:53.946668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:53.946955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/aa68451f072d4eeebdb68b90cad86309/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:53.951936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13, user_id=aa68451f072d4eeebdb68b90cad86309, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:53.953532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:53.972015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:53.980563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b18291-5dac-49ee-a96d-50264df46a79 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:53.981432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 125/231] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:53 2022] PUT /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/aa68451f072d4eeebdb68b90cad86309/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:17:54.027879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:54.029330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:54.029832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:54.030307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] PATH_INFO: `/v3/users/aa68451f072d4eeebdb68b90cad86309` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:54.031995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] RBAC: Authorizing `identity:get_user(user_id=aa68451f072d4eeebdb68b90cad86309)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:54.033705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:54.035645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 107/232] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:53 2022] GET /identity/v3/users/aa68451f072d4eeebdb68b90cad86309 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:54.041587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:54.074741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:54.170359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa95642-3ccd-406b-819b-1a4c77d8af58 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:54.227303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2152627-6b36-42db-a14a-4588b8100807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:54.228337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2152627-6b36-42db-a14a-4588b8100807 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:54.228863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2152627-6b36-42db-a14a-4588b8100807 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:54.229362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2152627-6b36-42db-a14a-4588b8100807 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:54.230793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2152627-6b36-42db-a14a-4588b8100807 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:54.232591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2152627-6b36-42db-a14a-4588b8100807 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:54.234497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 126/233] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:54 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:54.241114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:54.351105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf179a6-6384-43e1-bf4c-8806abafe2eb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:54.387611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0833a7-af84-45b7-b292-90db6549df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:54.388645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0833a7-af84-45b7-b292-90db6549df7d None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:54.389150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0833a7-af84-45b7-b292-90db6549df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:54.389627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0833a7-af84-45b7-b292-90db6549df7d None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/aa68451f072d4eeebdb68b90cad86309/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:54.392196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0833a7-af84-45b7-b292-90db6549df7d None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13, user_id=aa68451f072d4eeebdb68b90cad86309, role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:54.394012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0833a7-af84-45b7-b292-90db6549df7d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:54.420296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 108/234] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:54 2022] PUT /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/aa68451f072d4eeebdb68b90cad86309/roles/5a02bc464efd499bb3230455492613ae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:17:55.054489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d596d-e50b-4cbe-96d0-141a75af0562 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.054873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d596d-e50b-4cbe-96d0-141a75af0562 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.055066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d596d-e50b-4cbe-96d0-141a75af0562 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.056680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 127/235] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:55.062165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.062615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.062689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.092399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:55.093597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:55.117066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:55.132018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eb03c-1b5f-4360-95ce-3da770e5cb3d None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.132478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eb03c-1b5f-4360-95ce-3da770e5cb3d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.132478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eb03c-1b5f-4360-95ce-3da770e5cb3d None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.133933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 128/236] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:55.147325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.147613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.147886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.172809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:55.174004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:55.202699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 109/237] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:55.257716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:55.289501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 129/238] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:55.313902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:55.356698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:55.397722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406f9e-ef69-464a-882c-d64b6c66c07d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:55.441473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:55.442109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.442387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.442656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.454971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:55.460775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:55.461801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:17:55.462989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:55.467765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 110/239] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:55.469733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:55.488628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:55.586446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7bf89-f935-4ee9-84e8-eb8452371c98 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:55.621034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c5872-7eec-45dc-84b6-afba22a77f3e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:55.625296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:55.626714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.627277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.627744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.629621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:55.630132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:55.631752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:55.645366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 111/240] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:55.647830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:55.730254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:55.755881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:55.756530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.756809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.757079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.758715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a993c9f-50a8-4d9c-a361-325c1d54d9b8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:55.763858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:55.765341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:55.767115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 130/241] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:55.776077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:55.797230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:55.798926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.799583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.800146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] PATH_INFO: `/v3/users/fd70ec9596c149ce8463386aa0f813f7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.817782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] RBAC: Authorizing `identity:get_user(user_id=fd70ec9596c149ce8463386aa0f813f7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:55.819687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:55.821685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 112/242] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:55 2022] GET /identity/v3/users/fd70ec9596c149ce8463386aa0f813f7 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:55.827486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:55.951025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36f09f-53e1-452a-a311-1575f7e63a4f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:55.970146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:55.991340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbb59-3f47-41be-9c0f-2fc8603d4fae None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:55.992002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:55.993048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:55.993543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:55.994155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:55.999544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:56.057163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74b50-0aab-435d-8dbb-72b0c084bdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:56.057840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74b50-0aab-435d-8dbb-72b0c084bdec None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.058056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74b50-0aab-435d-8dbb-72b0c084bdec None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.058330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74b50-0aab-435d-8dbb-72b0c084bdec None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.058985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74b50-0aab-435d-8dbb-72b0c084bdec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:56.060473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74b50-0aab-435d-8dbb-72b0c084bdec None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:56.073077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:56.075227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:56.105663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 131/243] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:55 2022] POST /identity/v3/users => generated 293 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:17:56.177714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdab97-d807-4d9e-8636-d4673becc987 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.178400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdab97-d807-4d9e-8636-d4673becc987 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.178400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdab97-d807-4d9e-8636-d4673becc987 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.179604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 132/244] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:56.192673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.193073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.193197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.199408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 113/245] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:17:56.202976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:56.240278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:56.241491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:56.243465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:56.318453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa1abf-3be4-45a1-a910-5880aacd7b72 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:56.358573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:56.359799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.360310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.360841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.362326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:56.365180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:56.371228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 114/246] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:56.373722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:56.374977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 133/247] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:56.397893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:56.497068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:56.502246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2da25-4664-46e1-8bcd-7812e604eeee None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:56.507822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:56.539634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e07035-0b8d-441b-be00-427762faa9b8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:56.550715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:56.551825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.552120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.552399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.553151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] RBAC: Adding query filter params (role.id=ade6fa30e2cc403b821cd127af76ebd7, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=fd70ec9596c149ce8463386aa0f813f7) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:56.553464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:56.555264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6448931-0731-41c3-8e0e-54f6f2b42f1e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:56.566006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 115/248] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/role_assignments?user.id=fd70ec9596c149ce8463386aa0f813f7&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=ade6fa30e2cc403b821cd127af76ebd7 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:56.566196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcaa89-d146-4285-9c32-00db8691d0c2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.566476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcaa89-d146-4285-9c32-00db8691d0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.566635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcaa89-d146-4285-9c32-00db8691d0c2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.567947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 116/249] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:56.573392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.573804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.574184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.595146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:56.596373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:56.601649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:56.602316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.602600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.602879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.618766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:56.624429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:56.625377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:56.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 134/250] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:56.631940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:56.690328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 117/251] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:56.716241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:56.751929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193f96-3bb9-4166-abdc-f87c5fee84a4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:56.790717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:56.791409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.791692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.791968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.792840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:56.793070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:56.794755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:56.802209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:56.807446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 135/252] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:56.812810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:56.828240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:56.837510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b44521-b17d-448e-9c2b-305d2845d3db None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:56.948747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:56.949399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:56.949677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:56.949949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:56.971807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d45f01-a9c4-4f40-b785-784600fece39 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:56.983119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:56.987963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:56.988827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:17:56.990033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 118/253] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:17:56.995369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.040860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.042317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.042777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] PATH_INFO: `/v3/users/ad9bb028917c4eaa84443f62df561c9b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.047844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] RBAC: Authorizing `identity:get_user(user_id=ad9bb028917c4eaa84443f62df561c9b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:57.049541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:57.051490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 136/254] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/users/ad9bb028917c4eaa84443f62df561c9b => generated 297 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:57.057135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.185531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:57.251166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99224caf-900e-4a51-9175-6c936dc7bcce None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:57.281075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588a028-73e0-4b62-8381-c8501491e601 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:57.316937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.317474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.317756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.318030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.318701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:57.319008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:57.320553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed93700-269d-4086-ab47-136a614aefff None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:57.327908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.329379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.329831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.330175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 119/255] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:57.330489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.331907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:57.333536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:57.335467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 137/256] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:57 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:57.337432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6de48-dd09-44b5-ab49-f110b2e3d27f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.337432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6de48-dd09-44b5-ab49-f110b2e3d27f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.337432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6de48-dd09-44b5-ab49-f110b2e3d27f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.337768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 120/257] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:57.338730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.342548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6de48-dd09-44b5-ab49-f110b2e3d27f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.363528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:57.468656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd7af9-47c7-4e48-910e-8f0f565f8c98 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:57.501285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6de48-dd09-44b5-ab49-f110b2e3d27f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:57.521982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792b11b-4922-41ae-a2ae-428ee5b8e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.522909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792b11b-4922-41ae-a2ae-428ee5b8e13b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.523405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792b11b-4922-41ae-a2ae-428ee5b8e13b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.523901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792b11b-4922-41ae-a2ae-428ee5b8e13b None admin] PATH_INFO: `/v3/users/aa68451f072d4eeebdb68b90cad86309` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.527756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792b11b-4922-41ae-a2ae-428ee5b8e13b None admin] RBAC: Authorizing `identity:get_user(user_id=aa68451f072d4eeebdb68b90cad86309)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:57.529403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792b11b-4922-41ae-a2ae-428ee5b8e13b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:57.531198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 138/258] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:57 2022] GET /identity/v3/users/aa68451f072d4eeebdb68b90cad86309 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:57.532579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.533072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.533544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.540221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.540687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.540968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.541245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/ad9bb028917c4eaa84443f62df561c9b/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.543277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13, user_id=ad9bb028917c4eaa84443f62df561c9b, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:57.544718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:57.555431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:57.559580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 121/259] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:17:57 2022] PUT /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/ad9bb028917c4eaa84443f62df561c9b/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:17:57.560848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.576160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:57.577476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:57.651799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:57.681163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8330b-b451-44b5-9207-d97a17bcbd54 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:57.709566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 139/260] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:57.720510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.721181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.721462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.721748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:57.748003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.798205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:57.799773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:57.888028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:57.918497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 122/261] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:57.924255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:57.934373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d619d-a66c-47d8-a323-b8e0fe7cbf27 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:57.983722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:57.984361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:57.984637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:57.984956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.004892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:58.009626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:58.010495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:17:58.011684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:58.012730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:58.015156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 140/262] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:17:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:58.024157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:58.033425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81669e18-b6ad-42f1-b17d-e9faf96e20d3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:58.070924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:58.071613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:58.071898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:58.072158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.073282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:58.074743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:58.076620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 123/263] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:57 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:58.082757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:58.148569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:58.149678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f830c-c42f-437b-a486-326ba229c595 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:58.206252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79c64f-1784-4a8b-b034-b8f4f391f50d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:58.223800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:58.224453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:58.224692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:58.224957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.225627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:58.225926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:58.227417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:58.243614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:58.244364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:58.244751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:58.245060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 141/264] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:17:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:58.245364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.246197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=aa68451f072d4eeebdb68b90cad86309) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:58.246708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:58.248335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:58.252107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:58.255822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 124/265] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:58 2022] GET /identity/v3/role_assignments?user.id=aa68451f072d4eeebdb68b90cad86309&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:17:58.383487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:58.421494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61baab5-ad1d-4631-b18a-2256af32d316 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:58.457978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:58.458622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:58.458902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:58.459206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] PATH_INFO: `/v3/users/aa68451f072d4eeebdb68b90cad86309` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.460372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] RBAC: Authorizing `identity:get_user(user_id=aa68451f072d4eeebdb68b90cad86309)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:58.461826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:58.463756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 142/266] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:17:58 2022] GET /identity/v3/users/aa68451f072d4eeebdb68b90cad86309 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:58.471428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:58.594018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a651535f-98d8-4b28-952c-6bedc34634dd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:58.605109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:58.636795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:58.637423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:58.637702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:58.637974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.729245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:58.730655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:58.843010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 125/267] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:17:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:17:58.848472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:58.954937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6bd89-ac5e-419c-9619-0e0ca2532f55 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:58.979549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:58.990433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c858e2e-7c4a-4844-95d1-23ace4955dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:58.991071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858e2e-7c4a-4844-95d1-23ace4955dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:58.991379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858e2e-7c4a-4844-95d1-23ace4955dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:58.991654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c858e2e-7c4a-4844-95d1-23ace4955dc1 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:58.992915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c858e2e-7c4a-4844-95d1-23ace4955dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:58.994449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c858e2e-7c4a-4844-95d1-23ace4955dc1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:58.996330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 143/268] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:17:58 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:59.002718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:59.013640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:59.134095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75b5e5-484d-49c9-b6a2-4145632b5d47 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:59.173165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc09e-76ad-4d3d-b22c-a5888adc417e None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.173469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc09e-76ad-4d3d-b22c-a5888adc417e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.173710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdc09e-76ad-4d3d-b22c-a5888adc417e None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.175049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 144/269] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:17:59.180578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:59.181189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.181527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.181882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.182663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] RBAC: Adding query filter params (role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=aa68451f072d4eeebdb68b90cad86309) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:17:59.182985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:59.183172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d99e9-02b1-486c-9085-1a8bfb7a3146 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.183453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d99e9-02b1-486c-9085-1a8bfb7a3146 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.183738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d99e9-02b1-486c-9085-1a8bfb7a3146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.184499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:59.191923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 126/270] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:17:59 2022] GET /identity/v3/role_assignments?user.id=aa68451f072d4eeebdb68b90cad86309&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=5a02bc464efd499bb3230455492613ae => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:59.216950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-425d99e9-02b1-486c-9085-1a8bfb7a3146 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:59.218173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-425d99e9-02b1-486c-9085-1a8bfb7a3146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:59.328090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 145/271] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:17:59.353314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:59.455625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:59.465866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a43ae7-0c28-4582-b644-506aef9d680c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.466146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a43ae7-0c28-4582-b644-506aef9d680c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.466429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a43ae7-0c28-4582-b644-506aef9d680c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.468584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 146/272] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:59.474027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.474304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.474568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.494083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07639015-bb83-4fd5-a182-e203076f6959 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:59.497222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:59.498551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:59.538166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:59.539327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.539867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.540367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.542793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:59.544498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:59.546837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 127/273] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:17:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:17:59.554836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:59.595191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 147/274] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:59.614949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:59.683691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef4722-46b6-4a0c-8ec6-ac0d186ab3f4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:59.689267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:59.745298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10dbef-2fd6-4cb3-bc5d-9c414f01918c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:59.745924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10dbef-2fd6-4cb3-bc5d-9c414f01918c None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.746206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10dbef-2fd6-4cb3-bc5d-9c414f01918c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.746480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10dbef-2fd6-4cb3-bc5d-9c414f01918c None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.747161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10dbef-2fd6-4cb3-bc5d-9c414f01918c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:17:59.748954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10dbef-2fd6-4cb3-bc5d-9c414f01918c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:17:59.754431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff4657-0676-43e9-b582-f703fa5c4e53 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:17:59.766571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:17:59.782219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 128/275] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:17:59 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:17:59.791120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cabca5-bf05-40c1-9400-6a2229d99576 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.791120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cabca5-bf05-40c1-9400-6a2229d99576 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.791120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cabca5-bf05-40c1-9400-6a2229d99576 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.791120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 129/276] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:17:59.797974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.798315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.798638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.816622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:17:59.817797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:17:59.818263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:17:59.818717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:17:59.823843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:17:59.825093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:17:59.832885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:17:59.838697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:17:59.839798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:17:59.840774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:17:59.843139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 148/277] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:17:59.852007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:17:59.926157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 130/278] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:17:59.942912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.008320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7e9e9-ca9d-477e-bb06-16028be91281 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.039222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:00.074012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.074659 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.074936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.075239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.075977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:00.076283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:00.077876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:00.087499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030d9a-e987-464e-b4cd-6bff31b5562f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.095129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 149/279] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:17:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:00.097343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.143641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:00.162986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.163681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.163964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.165530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.180501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:00.185778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:00.186720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:00.187594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 131/280] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:17:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:00.193983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.248221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ef5f3d-21c2-4445-a1c3-bf9b662d030e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.293795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.294819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.295324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.295883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] PATH_INFO: `/v3/users/2bd959132f304f4ca2d5a565152e2522` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.311952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] RBAC: Authorizing `identity:get_user(user_id=2bd959132f304f4ca2d5a565152e2522)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:00.314572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:00.316642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 150/281] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:00 2022] GET /identity/v3/users/2bd959132f304f4ca2d5a565152e2522 => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:00.324140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.327950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de195-19c0-4e3e-b8db-7116bc9c4531 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.366778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:00.368045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.368618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.368903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.369244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.369904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:00.370210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:00.374366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:00.383934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 132/282] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:00.389513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.456539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:00.463477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6aa896-f08e-4481-aab9-302fcdcde50f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.511323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.512957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.513448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.514144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.515506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, user.id=2bd959132f304f4ca2d5a565152e2522, scope.system=all) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:00.516944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:00.518449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97364b-8012-4df1-9cb5-d5d2a3800d3b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.519185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a899276-02f8-4403-bfad-bef441b1d4de None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:00.527540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 151/283] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 14:18:00 2022] GET /identity/v3/role_assignments?user.id=2bd959132f304f4ca2d5a565152e2522&scope.system=all&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:00.529142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57577470-a476-44e1-ae59-dad3ae7407cc None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.529778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57577470-a476-44e1-ae59-dad3ae7407cc None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.530258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57577470-a476-44e1-ae59-dad3ae7407cc None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.531971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 152/284] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:00.542897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.542897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.542897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.568302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:00.569838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:00.606917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.607599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.607883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.608153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] PATH_INFO: `/v3/users/fd70ec9596c149ce8463386aa0f813f7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.610904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] RBAC: Authorizing `identity:get_user(user_id=fd70ec9596c149ce8463386aa0f813f7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:00.612658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:00.617126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 133/285] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:00 2022] GET /identity/v3/users/fd70ec9596c149ce8463386aa0f813f7 => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:00.626571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.731575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 153/286] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:00.749749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.779374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd51e0-6de4-4bc0-a104-5e25b5ab7d55 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.800598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:00.806792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:00.841528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.842668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.843830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.844961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:00.846374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:00.848822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:00.850836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 134/287] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:00 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:00.857807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:00.910679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d3bd1-5003-4121-91d3-d11f617cc94b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:00.990374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:00.990933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:00.991307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:00.991578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:01.010739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:01.015503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:01.016936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:01.017694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:01.025672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 154/288] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:01.026554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:01.044780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26a9f17-9e5a-45cb-88c2-dd5fc5a29b0c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:01.081435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:01.082239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:01.082521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:01.082794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/fd70ec9596c149ce8463386aa0f813f7/roles/ade6fa30e2cc403b821cd127af76ebd7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:01.085002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13, user_id=fd70ec9596c149ce8463386aa0f813f7, role_id=ade6fa30e2cc403b821cd127af76ebd7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:01.086955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:01.102970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 135/289] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:00 2022] PUT /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/users/fd70ec9596c149ce8463386aa0f813f7/roles/ade6fa30e2cc403b821cd127af76ebd7 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:01.159419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:01.176214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e858aa3-e562-41ea-9fa8-264dcb20873d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:01.217991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:01.218614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:01.218897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:01.219223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:01.219905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:01.220205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:01.221597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:01.231202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 155/290] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:01.237293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:01.321364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:01.386585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d024b2-79bd-4020-8027-4d6a3d843582 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:01.423378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:01.423785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:01.424068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:01.424345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] PATH_INFO: `/v3/users/ad9bb028917c4eaa84443f62df561c9b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:01.425764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] RBAC: Authorizing `identity:get_user(user_id=ad9bb028917c4eaa84443f62df561c9b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:01.427538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:01.429570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 136/291] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:01 2022] GET /identity/v3/users/ad9bb028917c4eaa84443f62df561c9b => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:01.435977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:01.556602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ee22c-778d-4091-a227-1693000a0546 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:01.599779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:01.600677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:01.601147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:01.601593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:01.626922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:01.696902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:01.698430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:01.825315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 156/292] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:01.831153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:01.971326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de4609-ba36-4525-a15a-2b231a736f93 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:01.989260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:02.016418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efade3-8b94-4277-ab7f-47a473a99cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:02.017058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efade3-8b94-4277-ab7f-47a473a99cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.017339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efade3-8b94-4277-ab7f-47a473a99cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.017614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efade3-8b94-4277-ab7f-47a473a99cc2 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.018737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efade3-8b94-4277-ab7f-47a473a99cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:02.020224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efade3-8b94-4277-ab7f-47a473a99cc2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:02.022176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 137/293] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:01 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:02.027999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:02.087248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:02.158061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165052a-bf05-44da-a5b7-f3742c7080b3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:02.194006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:02.194681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.194961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.195262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.195994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=ad9bb028917c4eaa84443f62df561c9b) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:02.196300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:02.197732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062471e-c8dd-496e-ac02-8cb22bbec8b0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:02.204231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 157/294] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:02 2022] GET /identity/v3/role_assignments?user.id=ad9bb028917c4eaa84443f62df561c9b&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:02.261311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17111dff-1c98-46f1-8d49-e2467dfb6682 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.261650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17111dff-1c98-46f1-8d49-e2467dfb6682 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.261870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17111dff-1c98-46f1-8d49-e2467dfb6682 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.263579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 138/295] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:02.269059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.269545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.269958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.289128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:02.290616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:02.375041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 158/296] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:02.408674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17111dff-1c98-46f1-8d49-e2467dfb6682 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:02.528181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17111dff-1c98-46f1-8d49-e2467dfb6682 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:02.566066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be743455-df2e-48eb-88b3-7df21704095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:02.566699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be743455-df2e-48eb-88b3-7df21704095e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.566979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be743455-df2e-48eb-88b3-7df21704095e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.567302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be743455-df2e-48eb-88b3-7df21704095e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.578855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be743455-df2e-48eb-88b3-7df21704095e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-be743455-df2e-48eb-88b3-7df21704095e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:02.584207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:02.585088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:02.585864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 139/297] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:02.591270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:02.659728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:02.681535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a8da5-e23f-46ca-b16c-127482445c69 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.681844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a8da5-e23f-46ca-b16c-127482445c69 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.682078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a8da5-e23f-46ca-b16c-127482445c69 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.683779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 140/298] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:02.690032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80837218-a841-4814-a409-4a9a051d7656 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.690377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80837218-a841-4814-a409-4a9a051d7656 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.690667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80837218-a841-4814-a409-4a9a051d7656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.718686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-80837218-a841-4814-a409-4a9a051d7656 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:02.720487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-80837218-a841-4814-a409-4a9a051d7656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:02.735507 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448cde54-14a3-4d34-83d7-424f2f54c7f2 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:02.766062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:02.777199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:02.778177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.778460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.778729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.779483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:02.779780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:02.781909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:02.791530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 159/299] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:02.797842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:02.806425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 141/300] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:02.829927 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80837218-a841-4814-a409-4a9a051d7656 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:02.929378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e07aad-6f19-4caf-b677-1e89486edf88 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:02.931913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:02.954422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80837218-a841-4814-a409-4a9a051d7656 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:02.976984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:02.977620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:02.977897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:02.978167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] PATH_INFO: `/v3/users/2d635ebed836426983b7a0cafefa6af1` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:02.993838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] RBAC: Authorizing `identity:get_user(user_id=2d635ebed836426983b7a0cafefa6af1)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:02.995664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:03.002317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.002317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.002317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.002317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.002658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:03.002967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:03.003198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 160/301] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:02 2022] GET /identity/v3/users/2d635ebed836426983b7a0cafefa6af1 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:03.004601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 142/302] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:03.005029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.013309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.058623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:03.128651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b44686-9bd2-4678-82fc-39f315189232 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:03.143614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e9e73-ccff-4479-9a88-2f33ab4e4fad None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:03.167017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9f482-e472-4c99-96e4-3c88850944da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.168078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9f482-e472-4c99-96e4-3c88850944da None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.168549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9f482-e472-4c99-96e4-3c88850944da None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.169035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9f482-e472-4c99-96e4-3c88850944da None admin] PATH_INFO: `/v3/groups` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.169847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9f482-e472-4c99-96e4-3c88850944da None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:03.171627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9f482-e472-4c99-96e4-3c88850944da None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:03.187319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.188408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.188710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.188986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.189843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] RBAC: Adding query filter params (role.id=ade6fa30e2cc403b821cd127af76ebd7, user.id=2d635ebed836426983b7a0cafefa6af1, scope.system=all) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:03.190197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:03.191681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0ba60-b140-4be7-873b-17a80fd3adec None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:03.192966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 143/303] 10.222.0.50 () {60 vars in 1134 bytes} [Mon Aug 8 14:18:03 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:18:03.202969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 161/304] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 14:18:03 2022] GET /identity/v3/role_assignments?user.id=2d635ebed836426983b7a0cafefa6af1&scope.system=all&role.id=ade6fa30e2cc403b821cd127af76ebd7 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:03.223194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f40713-24af-46e5-af53-935ef4c6817e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.223481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f40713-24af-46e5-af53-935ef4c6817e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.223759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f40713-24af-46e5-af53-935ef4c6817e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.225585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 144/305] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:03.231441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.231958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.232380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.254598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:03.256284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:03.283852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:03.342510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 162/306] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:03.365876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f40713-24af-46e5-af53-935ef4c6817e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.477513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:03.483009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f40713-24af-46e5-af53-935ef4c6817e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:03.521327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4288a03-537d-46a6-8f98-96e77de48bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.522027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288a03-537d-46a6-8f98-96e77de48bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.522302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288a03-537d-46a6-8f98-96e77de48bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.522638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4288a03-537d-46a6-8f98-96e77de48bf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.534514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4288a03-537d-46a6-8f98-96e77de48bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c4288a03-537d-46a6-8f98-96e77de48bf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:03.539376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:03.540236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:03.541396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 145/307] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:03.546349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.649549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3277-4563-45a4-8773-a143b471b007 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.650165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3277-4563-45a4-8773-a143b471b007 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.650448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3277-4563-45a4-8773-a143b471b007 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.652344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 146/308] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:03.652751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5d7b7-7818-4d98-bddd-acf4e92bdbc5 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:03.660268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.660573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.660842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.682674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:03.684199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:03.692863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.693871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.694388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.694716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.695533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:03.695887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:03.697328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:03.707130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 163/309] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:03.712877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.738606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:03.787659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 147/310] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:03.818317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.887674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9904cb0c-6c6e-4058-9f43-0c4eba652abf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:03.926229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:03.927849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.928497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.928968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.929063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] PATH_INFO: `/v3/users/2bd959132f304f4ca2d5a565152e2522` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.930326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] RBAC: Authorizing `identity:get_user(user_id=2bd959132f304f4ca2d5a565152e2522)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:03.931909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:03.933706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 164/311] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:03 2022] GET /identity/v3/users/2bd959132f304f4ca2d5a565152e2522 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:03.939702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:03.943335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215be1d5-c673-424b-8c88-d3a766a71410 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:03.981058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:03.982487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:03.982960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:03.983473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:03.998853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:04.006737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:04.007657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:04.008413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:04.011669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 148/312] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:04.018514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:04.037416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:04.054773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0a520-081c-4b0a-b175-787a7541e016 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:04.091065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:04.091753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:04.092035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:04.092313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] PATH_INFO: `/v3/system/users/2bd959132f304f4ca2d5a565152e2522/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:04.093818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2bd959132f304f4ca2d5a565152e2522, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:04.095232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:04.111351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 165/313] 10.222.0.50 () {58 vars in 1334 bytes} [Mon Aug 8 14:18:03 2022] PUT /identity/v3/system/users/2bd959132f304f4ca2d5a565152e2522/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:04.131905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f9c85-8d2d-4ebe-bdf5-d8af9356ec6c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:04.171413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:04.172445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:04.172917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:04.173377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:04.174277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:04.174779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:04.176429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:04.186513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 149/314] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:04.193922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:04.315580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272312dd-c6e1-4fd1-a404-af3e9b5f4ba9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:04.358340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:04.359901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:04.360397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:04.360861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] PATH_INFO: `/v3/users/fd70ec9596c149ce8463386aa0f813f7` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:04.362200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] RBAC: Authorizing `identity:get_user(user_id=fd70ec9596c149ce8463386aa0f813f7)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:04.363880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:04.365722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 166/315] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:04 2022] GET /identity/v3/users/fd70ec9596c149ce8463386aa0f813f7 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:04.371665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:04.391475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:04.482820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf903f9-ad8c-4a13-832c-4c1c85437c99 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:04.519200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:04.519803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:04.520080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:04.520350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:04.590120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:04.591623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:04.604085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:04.694111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 150/316] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:04.700112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:04.721737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:04.807355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5647b4-9de1-4a1f-a585-d32661228559 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:04.844501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f72a1-b09b-47dc-a500-404c06e24ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:04.845056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f72a1-b09b-47dc-a500-404c06e24ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:04.845333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f72a1-b09b-47dc-a500-404c06e24ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:04.845603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f72a1-b09b-47dc-a500-404c06e24ab4 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:04.846727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f72a1-b09b-47dc-a500-404c06e24ab4 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:04.848206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f72a1-b09b-47dc-a500-404c06e24ab4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:04.854635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 167/317] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:04 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:04.856076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:04.971149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4841d-ce48-42d9-9a9c-8afd919f1562 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:04.982765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:05.008151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:05.008758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.009036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.009303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.010029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] RBAC: Adding query filter params (role.id=ade6fa30e2cc403b821cd127af76ebd7, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13, user.id=fd70ec9596c149ce8463386aa0f813f7) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:05.010333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:05.011836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a5e08-721e-4b7b-b9e6-873997ae484c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:05.023217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 151/318] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:04 2022] GET /identity/v3/role_assignments?user.id=fd70ec9596c149ce8463386aa0f813f7&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=ade6fa30e2cc403b821cd127af76ebd7 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:05.296200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2c24-34b0-4f35-b9cc-ff95eacd1d87 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.296389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2c24-34b0-4f35-b9cc-ff95eacd1d87 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.296786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa2c24-34b0-4f35-b9cc-ff95eacd1d87 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.298953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 168/319] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:05.303831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.304123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.304398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.323581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:05.324786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:05.394846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 152/320] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:05.411182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4ad15-0dee-4390-9c92-a1a31118fda4 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.411487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4ad15-0dee-4390-9c92-a1a31118fda4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.411749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a4ad15-0dee-4390-9c92-a1a31118fda4 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.413228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 169/321] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:05.415910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:05.417781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.418081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.418360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.438570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:05.439780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:05.516726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 170/322] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:05.525935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac0c30-98dd-4745-ac6f-7c48bcb5fb56 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:05.529084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:05.538856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:05.557888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:05.566289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:05.566961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.567278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.567553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.578444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:05.583274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:05.584192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:05.585050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 153/323] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:05.590038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:05.647384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2ea61-10c4-4b67-8e50-6b7d5e7bb1d4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:05.682577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:05.683406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.683687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.684017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.685156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:05.686861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:05.688518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 171/324] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:05.694876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:05.699679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8b589-d29f-4192-902b-8e4581e472a7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:05.736502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:05.737112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.737390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.738064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.738865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:05.739405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:05.740811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:05.749565 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 154/325] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:05.754846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:05.784766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:05.788650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:05.806243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65345490-d8c2-45ce-af47-adda19caabdb None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:05.846986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:05.847738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.848016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.848373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] PATH_INFO: `/v3/groups` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.848908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:05.850341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:05.870128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 172/326] 10.222.0.50 () {60 vars in 1134 bytes} [Mon Aug 8 14:18:05 2022] POST /identity/v3/groups => generated 226 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:18:05.902691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908394d8-b552-4ce6-9ca7-fd46aad2b351 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:05.944698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024d4fc-df3d-4664-b857-95b0bebe2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:05.945346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024d4fc-df3d-4664-b857-95b0bebe2756 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:05.945626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024d4fc-df3d-4664-b857-95b0bebe2756 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:05.945895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024d4fc-df3d-4664-b857-95b0bebe2756 None admin] PATH_INFO: `/v3/users/2d635ebed836426983b7a0cafefa6af1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:05.947175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024d4fc-df3d-4664-b857-95b0bebe2756 None admin] RBAC: Authorizing `identity:get_user(user_id=2d635ebed836426983b7a0cafefa6af1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:05.948658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c024d4fc-df3d-4664-b857-95b0bebe2756 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:05.950548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 155/327] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:05 2022] GET /identity/v3/users/2d635ebed836426983b7a0cafefa6af1 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:05.956974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:06.066246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bfa31-fe9c-4f78-9126-b0872211f042 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:06.101431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea7bc5-71a4-47ec-9c2b-0d62e5aa9b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:06.102021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea7bc5-71a4-47ec-9c2b-0d62e5aa9b09 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:06.102302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea7bc5-71a4-47ec-9c2b-0d62e5aa9b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:06.102579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea7bc5-71a4-47ec-9c2b-0d62e5aa9b09 None admin] PATH_INFO: `/v3/system/users/2d635ebed836426983b7a0cafefa6af1/roles/ade6fa30e2cc403b821cd127af76ebd7` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:06.104049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea7bc5-71a4-47ec-9c2b-0d62e5aa9b09 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d635ebed836426983b7a0cafefa6af1, role_id=ade6fa30e2cc403b821cd127af76ebd7)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:06.105432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea7bc5-71a4-47ec-9c2b-0d62e5aa9b09 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:06.112692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 173/328] 10.222.0.50 () {58 vars in 1334 bytes} [Mon Aug 8 14:18:05 2022] PUT /identity/v3/system/users/2d635ebed836426983b7a0cafefa6af1/roles/ade6fa30e2cc403b821cd127af76ebd7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:06.156899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efd814-2e1e-40b7-8186-077e1b67d985 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:06.157745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efd814-2e1e-40b7-8186-077e1b67d985 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:06.158021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efd814-2e1e-40b7-8186-077e1b67d985 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:06.159622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 156/329] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:06.165666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:06.165958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:06.166236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:06.188049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:06.189308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:06.191896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:06.276649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 174/330] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:06.299736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efd814-2e1e-40b7-8186-077e1b67d985 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:06.414528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:06.429788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efd814-2e1e-40b7-8186-077e1b67d985 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:06.469543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:06.470531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:06.470984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:06.471465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:06.483522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:06.488711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:06.489551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:06.490329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:06.492598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 157/331] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:06.498047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:06.611346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8caf6-4011-4e97-956b-30b7a3b5661c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:06.655633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:06.656529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:06.656943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:06.657352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:06.658185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:06.658624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:06.660162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:06.666400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:06.672779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 175/332] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:06.677647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:06.783702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a327f88-2aa6-44a0-b81c-415f232e553c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:06.821734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fee6c6-c5a3-4ad5-be8a-958805e66dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:06.822401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fee6c6-c5a3-4ad5-be8a-958805e66dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:06.822685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fee6c6-c5a3-4ad5-be8a-958805e66dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:06.822964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fee6c6-c5a3-4ad5-be8a-958805e66dd3 None admin] PATH_INFO: `/v3/users/2bd959132f304f4ca2d5a565152e2522` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:06.824142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fee6c6-c5a3-4ad5-be8a-958805e66dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=2bd959132f304f4ca2d5a565152e2522)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:06.825598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fee6c6-c5a3-4ad5-be8a-958805e66dd3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:06.827933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 158/333] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:06 2022] GET /identity/v3/users/2bd959132f304f4ca2d5a565152e2522 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:06.833512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:06.971450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18469e57-0075-45f4-ae3c-13908cdc3eaf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:07.016631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:07.017600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:07.018187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:07.018670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:07.019635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] RBAC: Adding query filter params (role.id=575750f3d1c5499bbbb22af90adfb176, user.id=2bd959132f304f4ca2d5a565152e2522, scope.system=all) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:07.023689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:07.025373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ac55d-90da-4d51-a815-e3cce912e4ed None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:07.039777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 176/334] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 14:18:06 2022] GET /identity/v3/role_assignments?user.id=2bd959132f304f4ca2d5a565152e2522&scope.system=all&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:08.280774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aef0f0-f732-4476-9e22-e5f6df2455b5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.281297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aef0f0-f732-4476-9e22-e5f6df2455b5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.281297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aef0f0-f732-4476-9e22-e5f6df2455b5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:08.282849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 159/335] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:08.289291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.290189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.290660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:08.311566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:08.313670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:08.379764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:08.398381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 177/336] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:08.423808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aef0f0-f732-4476-9e22-e5f6df2455b5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:08.457807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:08.555214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aef0f0-f732-4476-9e22-e5f6df2455b5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:08.596187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:08.596846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.597125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.597398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:08.615132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:08.620663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:08.621544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:08.622461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 160/337] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:08.628580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:08.754384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bf5b5-f607-46ee-9f0d-3f77fb3af75a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:08.800918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:08.801551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.801830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.802103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:08.802785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:08.803102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:08.804573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7937790-9b52-4f58-8032-41be63aad330 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:08.816818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 178/338] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:08.822587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:08.914693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:08.940772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17a9b-80d6-4d2c-a7f5-d943ece7f626 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:08.981364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:08.981982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.982261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.982601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:08.991710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abde4ab-0104-4555-8989-c3a6451f1e95 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.992012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abde4ab-0104-4555-8989-c3a6451f1e95 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.992267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abde4ab-0104-4555-8989-c3a6451f1e95 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:08.993671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 179/339] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:08.998910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02856ead-098e-4530-8c82-91a00bb2af53 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:08.999223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02856ead-098e-4530-8c82-91a00bb2af53 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:08.999500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02856ead-098e-4530-8c82-91a00bb2af53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.032094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-02856ead-098e-4530-8c82-91a00bb2af53 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:09.033300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-02856ead-098e-4530-8c82-91a00bb2af53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:09.110568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:09.112092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:09.141393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:09.182479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 180/340] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:09.191121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019162a-1640-4be9-a73b-8b9cb27c8796 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.191407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019162a-1640-4be9-a73b-8b9cb27c8796 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.191681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019162a-1640-4be9-a73b-8b9cb27c8796 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.193265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 181/341] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:09.198379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b659985-a5b8-4261-a0d6-9babf1fc8190 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.198664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b659985-a5b8-4261-a0d6-9babf1fc8190 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.198939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b659985-a5b8-4261-a0d6-9babf1fc8190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.239328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4b659985-a5b8-4261-a0d6-9babf1fc8190 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:09.240529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4b659985-a5b8-4261-a0d6-9babf1fc8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:09.244537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:09.281931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 161/342] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:09.281931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.354139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 182/343] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:09.356222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598e741-da33-4d99-85e5-bd800e8a0c89 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.356619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598e741-da33-4d99-85e5-bd800e8a0c89 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.356995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598e741-da33-4d99-85e5-bd800e8a0c89 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.358651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 183/344] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:09.363231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598e741-da33-4d99-85e5-bd800e8a0c89 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.416896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e552d4-c6b6-46cb-ba23-6087e22cd237 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:09.464077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef7d92-ef03-4b09-86f4-b4cc11fb8073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:09.464982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef7d92-ef03-4b09-86f4-b4cc11fb8073 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.465388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef7d92-ef03-4b09-86f4-b4cc11fb8073 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.465793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef7d92-ef03-4b09-86f4-b4cc11fb8073 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.477412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef7d92-ef03-4b09-86f4-b4cc11fb8073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-18ef7d92-ef03-4b09-86f4-b4cc11fb8073 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:09.482440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:09.483473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:09.484383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:09.486384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 162/345] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:09.487802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.488285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.488727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.496035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598e741-da33-4d99-85e5-bd800e8a0c89 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:09.508492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:09.508948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:09.510214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:09.534769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:09.535460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.535748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.536027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.537171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:09.539822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:09.541980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 184/346] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:09.542599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.555932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:09.591638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 163/347] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:09.592773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.660929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901447ab-90e3-4707-a537-a52dce1e3c13 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:09.707551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:09.708494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.708783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.709058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.710634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bd78b-e95c-4a2d-9291-c215b8645088 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:09.722166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:09.728165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:09.729071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:09.729856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 185/348] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:09.730990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.756175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:09.757224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.757750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.758944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.759945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:09.760433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:09.762063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:09.772294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 164/349] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:09.773549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.783449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:09.843600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:09.875589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac32cdd0-4340-414b-abc9-1a653e2f2c93 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:09.920467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:09.921058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.921333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.922153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.934344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a71ba-6929-4a24-8a2e-7795e63acc98 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:09.938702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:09.941592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:09.947247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 186/350] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:09.948688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:09.977099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:09.977741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:09.978022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:09.978295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:09.993870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:10.001433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:10.002360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:10.003173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 165/351] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:10.004644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.081061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a360e3-2e29-4d57-a34b-2c1f3a93f098 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.082772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:10.126867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:10.128356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.129113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.129593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.129870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.131162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:10.131475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.133961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:10.135495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dbd53-4c63-4ed9-a62b-28ce9f18c233 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.144127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 187/352] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:10.144999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.181771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.182465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.182814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.183183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.276327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b3c93-3399-412e-b413-d7228a4cf4c1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.282388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.283924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:10.322855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.323558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.323841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.324118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.324866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=e319c2beee9242a98bbbec05ac9e6717) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:10.325180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.326641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:10.333229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 188/353] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:10.333907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.407529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:10.466479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:10.478827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 166/354] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:10.479789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.552365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83058ab-bcc7-4700-be19-45249ecc46b9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.612002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.612639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.612922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.613195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.613891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:10.614194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.616857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:10.632087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 189/355] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:10.632785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.641953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96554fef-7591-4d16-b053-36b4f6ab3463 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.682717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f649b-048e-4fb2-ae73-4d1b74aafafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.683768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f649b-048e-4fb2-ae73-4d1b74aafafc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.684250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f649b-048e-4fb2-ae73-4d1b74aafafc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.684718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f649b-048e-4fb2-ae73-4d1b74aafafc None admin] PATH_INFO: `/v3/users/2d635ebed836426983b7a0cafefa6af1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.686409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f649b-048e-4fb2-ae73-4d1b74aafafc None admin] RBAC: Authorizing `identity:get_user(user_id=2d635ebed836426983b7a0cafefa6af1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.688207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f649b-048e-4fb2-ae73-4d1b74aafafc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:10.690465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 167/356] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/users/2d635ebed836426983b7a0cafefa6af1 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:10.691937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f93ae-8a53-4e21-9ff1-c695924869bf None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.692424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f93ae-8a53-4e21-9ff1-c695924869bf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.692886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f93ae-8a53-4e21-9ff1-c695924869bf None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.694367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 168/357] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:10.695319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f93ae-8a53-4e21-9ff1-c695924869bf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.770408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:10.780056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:10.791325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dee829-16c9-4a13-8c5f-403440304c6c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.825246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f93ae-8a53-4e21-9ff1-c695924869bf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.838769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.840034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.840138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.842656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.842656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.843946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:10.846445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 190/358] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:10.847465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:10.874037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca0ae8-32b2-4438-a07b-94e55c10b705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:10.875377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca0ae8-32b2-4438-a07b-94e55c10b705 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:10.875764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca0ae8-32b2-4438-a07b-94e55c10b705 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:10.876412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca0ae8-32b2-4438-a07b-94e55c10b705 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:10.973223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e62ad7-d615-49c5-89f9-aaed986d0f6e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:10.979962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca0ae8-32b2-4438-a07b-94e55c10b705 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:10.985765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca0ae8-32b2-4438-a07b-94e55c10b705 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.021640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.022528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.022528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.022703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.023469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] RBAC: Adding query filter params (role.id=ade6fa30e2cc403b821cd127af76ebd7, user.id=2d635ebed836426983b7a0cafefa6af1, scope.system=all) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:11.023786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:11.025221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40aaef-112c-4648-a656-06aa3b95df79 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.040130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 191/359] 10.222.0.50 () {56 vars in 1308 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/role_assignments?user.id=2d635ebed836426983b7a0cafefa6af1&scope.system=all&role.id=ade6fa30e2cc403b821cd127af76ebd7 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:11.041491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.041703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.041935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.062595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:11.063918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:11.079405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:11.100844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:11.159610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 169/360] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:11.164562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7ef7-afe3-4670-9d5d-590504368ad4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.164830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7ef7-afe3-4670-9d5d-590504368ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.165108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c7ef7-afe3-4670-9d5d-590504368ad4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.166519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 170/361] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:11.168761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c7ef7-afe3-4670-9d5d-590504368ad4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.186638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 192/362] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:11.188188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.332422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c7ef7-afe3-4670-9d5d-590504368ad4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.332930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309a729-8793-4639-9197-67f071ac1ea0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.379433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff07c16-72e6-4ba4-bb9f-e30388ef5689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.380084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff07c16-72e6-4ba4-bb9f-e30388ef5689 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.380361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff07c16-72e6-4ba4-bb9f-e30388ef5689 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.380632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff07c16-72e6-4ba4-bb9f-e30388ef5689 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.381788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff07c16-72e6-4ba4-bb9f-e30388ef5689 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:11.382620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.383397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff07c16-72e6-4ba4-bb9f-e30388ef5689 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.383782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.384299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.384763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.389559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 193/363] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:11.389559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.389559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.389559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.389853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:11.390014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.391608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 171/364] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:11.392365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.410376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:11.411601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:11.424887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:11.431180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:11.495594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 194/365] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:11.496220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.518779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7be9b-ca52-4dee-9399-f16018be9c9c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.565601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.566243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.566525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.566826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.579983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:11.586623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:11.587661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:11.588454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 172/366] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:11.590682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.627932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222cd2-b1ee-43fd-a017-9f6a6feff1e8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.678952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.681858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.682332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.682796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.686271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:11.686271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.687425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 195/367] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:11.688683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.707258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:11.713426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:11.725046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ad6ab-18ca-410a-86ff-fbc9fcc6bf7a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.769938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.770549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.770852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.771121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.771859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=e319c2beee9242a98bbbec05ac9e6717) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:11.772166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:11.773955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.779646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 173/368] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=5a02bc464efd499bb3230455492613ae => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:11.780460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.839744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c3985-c069-4389-8b4e-65be5774a021 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.894784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.895753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.896224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.896775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.910889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:11.916804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:11.917718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:11.918473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:11.921159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 196/369] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:11.922325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:11.925175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5dbb9-fc2a-4b6b-a57e-505af3d49ed8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:11.981190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:11.982622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:11.983135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:11.983695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:11.985045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:11.985516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:11.987068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:11.998770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 174/370] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:12.000209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:12.022838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:12.027482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:12.063615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7571335-233d-4d47-852b-b02e8d27cf69 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:12.112237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:12.112941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.113315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.113734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.114560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:12.115012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:12.116700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:12.123209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 197/371] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:12.124231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:12.136010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57643f-dbfe-4d6f-8b5c-a47ac6fc2f8a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:12.184781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:12.185758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.186828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.187288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.188025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:12.188344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:12.190531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:12.207779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 175/372] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:12.210425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:12.291294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6c571-9fa0-4ba4-b58c-6cb4914312e2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:12.333137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:12.350558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:12.351739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.352036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.352318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.359921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:12.409513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80a274-31a2-4bb7-a1f9-045872b933ff None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:12.456128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:12.459308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:12.486330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:12.487328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.487830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.488318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.622395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:12.624280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:12.662370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 198/373] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:12.671308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:12.728861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:12.755761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 176/374] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:18:12.763633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:12.775874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:12.868844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073f878-8fc0-4e99-b37a-1b64576f21f2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:12.893108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85ee2d-c33b-4d0f-a95c-7d6faaef4653 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:12.922023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac781e55-dbf6-4d06-8d17-0ab13fe02cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:12.922982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac781e55-dbf6-4d06-8d17-0ab13fe02cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.923576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac781e55-dbf6-4d06-8d17-0ab13fe02cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.924055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac781e55-dbf6-4d06-8d17-0ab13fe02cc9 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.925588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac781e55-dbf6-4d06-8d17-0ab13fe02cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:12.927670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac781e55-dbf6-4d06-8d17-0ab13fe02cc9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:12.931359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 199/375] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:12 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:12.932829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cc563-1913-47c2-ab56-4730a6f5e814 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.933311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cc563-1913-47c2-ab56-4730a6f5e814 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.933775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cc563-1913-47c2-ab56-4730a6f5e814 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.938271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 200/376] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:12.938720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54df8d-e138-47b7-ac0b-435ecce596f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:12.939413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54df8d-e138-47b7-ac0b-435ecce596f6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.939697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54df8d-e138-47b7-ac0b-435ecce596f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.939974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54df8d-e138-47b7-ac0b-435ecce596f6 None admin] PATH_INFO: `/v3/projects/20988f0d51544772b78672526cf109cb` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.940090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cc563-1913-47c2-ab56-4730a6f5e814 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:12.943140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54df8d-e138-47b7-ac0b-435ecce596f6 None admin] RBAC: Authorizing `identity:get_project(project_id=20988f0d51544772b78672526cf109cb)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:12.943140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54df8d-e138-47b7-ac0b-435ecce596f6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:12.944572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 177/377] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:12 2022] GET /identity/v3/projects/20988f0d51544772b78672526cf109cb => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:12.948386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:12.948700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:12.948942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:12.972652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:12.974467 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:13.025626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:13.065226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 178/378] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:13.066523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:13.073904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cc563-1913-47c2-ab56-4730a6f5e814 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.077913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:13.122418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.123957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.124245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.124566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.128188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:13.129677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:13.131956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 201/379] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:12 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:13.132740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:13.219578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53345d0f-5511-4338-a606-444ef2d6aa3b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.267238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e181c6-e1b6-4840-b1db-2daf2be107ba None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.271149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.271997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.272417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.272827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] PATH_INFO: `/v3/groups/71855f429d984031a65d64e0d9f75dbf` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.283359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] RBAC: Authorizing `identity:get_group(group_id=71855f429d984031a65d64e0d9f75dbf)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:13.285351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:13.288321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 179/380] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/groups/71855f429d984031a65d64e0d9f75dbf => generated 229 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:13.289774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:13.314108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.315281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.315661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.316030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.331503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:13.337927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:13.338868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 202/381] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:13.343993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:13.353334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:13.383831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:13.434439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4d42-581f-4bf8-a31f-4a5e0a05890d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.478179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.478788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.479150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.479506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.480322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:13.480697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:13.482305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:13.493209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 180/382] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=5a02bc464efd499bb3230455492613ae => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:13.493209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:13.504919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1423a-2caa-425c-947f-d9ae267686c2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.572613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.574123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.575217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.575782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.577272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] RBAC: Adding query filter params (group.id=71855f429d984031a65d64e0d9f75dbf, role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=20988f0d51544772b78672526cf109cb) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:13.578462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:13.582240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:13.595288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 203/383] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/role_assignments?group.id=71855f429d984031a65d64e0d9f75dbf&scope.project.id=20988f0d51544772b78672526cf109cb&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 260 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:13.686217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10a0cb-b984-4d21-9fdc-2f6ebb05194b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.692483 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:13.744112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.744998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.745439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.745850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.746711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:13.747187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:13.748817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:13.759546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 181/384] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:13.766570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:13.905372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:13.913495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6825225-888b-46fe-9112-52cb2aec1ef1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:13.963908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:13.964586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:13.964867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:13.965139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:13.966374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:13.967850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:13.969799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 204/385] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:13.975169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:14.142394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ad23b-bfd6-403c-9481-348e55833e6e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:14.187997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a1b6e-123b-4152-8785-e076f67f3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:14.188638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a1b6e-123b-4152-8785-e076f67f3b7c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:14.188920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a1b6e-123b-4152-8785-e076f67f3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:14.189194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a1b6e-123b-4152-8785-e076f67f3b7c None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:14.190364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a1b6e-123b-4152-8785-e076f67f3b7c None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:14.191848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a1b6e-123b-4152-8785-e076f67f3b7c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:14.193811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 182/386] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:13 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:14.200074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:14.292877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe101e4-9a95-4645-8b1b-a33bb293ec0d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:14.293221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe101e4-9a95-4645-8b1b-a33bb293ec0d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:14.293431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe101e4-9a95-4645-8b1b-a33bb293ec0d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:14.294996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 183/387] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:14.301784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:14.302425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:14.303059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:14.352288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:14.353824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:14.395976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb906fa7-f2ac-42ca-974a-cae1a31767cd None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:14.415509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:14.446760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:14.447468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:14.447796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:14.448073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717/groups/633680b351a2424dbde8731f7438ea18/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:14.451330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c2beee9242a98bbbec05ac9e6717, group_id=633680b351a2424dbde8731f7438ea18, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:14.452831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:14.475344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 205/388] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 14:18:14 2022] PUT /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717/groups/633680b351a2424dbde8731f7438ea18/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:18:14.476566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 184/389] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:14.503464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:14.540763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:14.646913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658cc30-0635-4641-bbce-232e85dfc61d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:14.699899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:14.700967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:14.701508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:14.701988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:14.715876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:14.723308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:14.724283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:14.725220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:14.728044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 206/390] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:14.734672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:14.880844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ca953-65a7-4fe7-9f1e-5f88f46e5fea None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:14.920599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:14.921148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:14.921461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:14.921737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:14.922423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:14.922731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:14.924199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4cc28c-5639-4e1b-8f50-46e74a0b0b2b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:14.934401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 185/391] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:14.940353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:15.076971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52a5a5-760f-4003-b5e0-96a90ea8508c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.077300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52a5a5-760f-4003-b5e0-96a90ea8508c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.078080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52a5a5-760f-4003-b5e0-96a90ea8508c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.080568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 186/392] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:15.086063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.086357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.086625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.104703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:15.108366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:15.109832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:15.135595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6c9a-eadd-45de-aee0-f96079485a95 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:15.139308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:15.202294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 187/393] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:15.228422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:15.239982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:15.240662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.240947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.241234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.242631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:15.244188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:15.246955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 207/394] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:14 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:15.252261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:15.359511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4a70c-3718-45c7-96e9-48eb94187946 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:15.404070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:15.405007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.406863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.407336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.434019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:15.440485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:15.442529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:15.443559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 188/395] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:15.452145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:15.498456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:15.523372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059a786-b186-4a29-9d6e-6dbc3a7d1bae None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:15.600776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:15.601439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.601697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.601975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.603172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:15.604650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:15.609210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 208/396] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:15 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:15.616161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:15.650518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f6a92-7063-4c99-b462-5b8211f7df67 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:15.655464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:15.716157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:15.716780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.717057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.717332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.718002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] RBAC: Adding query filter params (name=member) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:15.718443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:15.719878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:15.737552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 189/397] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:15.752314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:15.803646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42012f99-c215-4f30-b4fc-561c247ee6db None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:15.887816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:15.888849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:15.889210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:15.889611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717/groups/633680b351a2424dbde8731f7438ea18/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:15.897491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e319c2beee9242a98bbbec05ac9e6717, group_id=633680b351a2424dbde8731f7438ea18, role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:15.898961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:15.938847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 209/398] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 14:18:15 2022] PUT /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717/groups/633680b351a2424dbde8731f7438ea18/roles/5a02bc464efd499bb3230455492613ae => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:15.949129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:15.977309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ba8e-2518-4153-a8df-77036a4256fe None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:16.018367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:16.018946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.019256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.019534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.020831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:16.022433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:16.024524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 190/399] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:15 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:16.030226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:16.137852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:16.165228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238bb53-5249-41ef-9d4a-becece2b7ab8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:16.209413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704f4c-de78-43c2-9704-ffc53498d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:16.210055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704f4c-de78-43c2-9704-ffc53498d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.210331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704f4c-de78-43c2-9704-ffc53498d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.210603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704f4c-de78-43c2-9704-ffc53498d8ce None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.211942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704f4c-de78-43c2-9704-ffc53498d8ce None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:16.213401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e704f4c-de78-43c2-9704-ffc53498d8ce None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:16.215463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 210/400] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:16 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:16.220961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:16.345907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f868-ed5f-4cfc-8367-ee902ee1394d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:16.348436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433aa5e-f734-4b5e-8d9c-8a08de971ba5 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.348632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433aa5e-f734-4b5e-8d9c-8a08de971ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.348881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433aa5e-f734-4b5e-8d9c-8a08de971ba5 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.350466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 211/401] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:16.367125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.367125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.367125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.396320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:16.397562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:16.402890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23baeb-3e53-4d8f-8235-1e1803df14ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:16.403604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23baeb-3e53-4d8f-8235-1e1803df14ea None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.403886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23baeb-3e53-4d8f-8235-1e1803df14ea None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.404162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23baeb-3e53-4d8f-8235-1e1803df14ea None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/groups/633680b351a2424dbde8731f7438ea18/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.407138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23baeb-3e53-4d8f-8235-1e1803df14ea None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13, group_id=633680b351a2424dbde8731f7438ea18, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:16.408580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23baeb-3e53-4d8f-8235-1e1803df14ea None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:16.433603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 191/402] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 14:18:16 2022] PUT /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/groups/633680b351a2424dbde8731f7438ea18/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:16.477970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553269-bf51-48b3-8f3d-564aaa5d54d0 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.478292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553269-bf51-48b3-8f3d-564aaa5d54d0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.478534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553269-bf51-48b3-8f3d-564aaa5d54d0 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.480324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 192/403] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:16.486731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.487061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.487379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.503520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:16.514277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:16.515902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:16.518802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 212/404] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:16.558940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:16.669275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 193/405] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:16.702260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:16.710417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:16.728354 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b5431-9d11-48a8-92c6-e6f5300434bd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:16.775564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:16.776187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.776464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.776802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.791663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:16.796566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:16.797497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:16.798272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 213/406] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:16.803581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:16.826550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924a2f7-8eb6-427c-bc71-06b9402a1774 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:16.879640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:16.881978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:16.882515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:16.883005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:16.889324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:16.907969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:16.913178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:16.914051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:16.914828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:16.924579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 194/407] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:16.930664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.045082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f646c30-df9e-412a-9211-bb0a8409e614 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.061377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9279c-6a13-4922-a172-6f8ac0c72c5c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.100155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.101347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.102146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.102631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.103534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:17.104051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.105645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.115881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 195/408] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:17.121805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.152506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.153187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.153403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.153674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.154351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:17.154652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.158482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.169583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:17.177813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 214/409] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:17.182944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.203156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:17.246936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8d148-8fc1-4a57-9e6d-269202736fa5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.288254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.289296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.289776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.290260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.291826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.293694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.295893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 196/410] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:17 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:17.302456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.393738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2897fa7-d4c0-4948-ab19-73778590bf76 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.441751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896b964-907f-49ab-b1f9-1e43c7096243 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.456836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcaa51-4a4b-4cab-bb18-fbef0fc3e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.458567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcaa51-4a4b-4cab-bb18-fbef0fc3e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.459054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcaa51-4a4b-4cab-bb18-fbef0fc3e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.459822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcaa51-4a4b-4cab-bb18-fbef0fc3e5d9 None admin] PATH_INFO: `/v3/groups/71855f429d984031a65d64e0d9f75dbf` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.461484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcaa51-4a4b-4cab-bb18-fbef0fc3e5d9 None admin] RBAC: Authorizing `identity:get_group(group_id=71855f429d984031a65d64e0d9f75dbf)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.463646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcaa51-4a4b-4cab-bb18-fbef0fc3e5d9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.466575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 215/411] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:17 2022] GET /identity/v3/groups/71855f429d984031a65d64e0d9f75dbf => generated 229 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:17.468514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343e8c-d1d2-41a6-a782-ec9121cdb7e6 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.468981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343e8c-d1d2-41a6-a782-ec9121cdb7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.469902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61343e8c-d1d2-41a6-a782-ec9121cdb7e6 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.472213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:17.472638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 216/412] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:17.475068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61343e8c-d1d2-41a6-a782-ec9121cdb7e6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.482832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938044a-d426-4e27-a10f-32d94e221a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.484141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938044a-d426-4e27-a10f-32d94e221a05 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.484496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938044a-d426-4e27-a10f-32d94e221a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.484842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938044a-d426-4e27-a10f-32d94e221a05 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.486075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938044a-d426-4e27-a10f-32d94e221a05 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.487718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938044a-d426-4e27-a10f-32d94e221a05 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.489794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 197/413] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:17 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:17.495827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.496179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.496550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.515989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:17.517285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:17.566969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:17.604228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61343e8c-d1d2-41a6-a782-ec9121cdb7e6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.605018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 198/414] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:17.606392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.641101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.641838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.642208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.642554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] PATH_INFO: `/v3/projects/20988f0d51544772b78672526cf109cb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.643793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] RBAC: Authorizing `identity:get_project(project_id=20988f0d51544772b78672526cf109cb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.645379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.647317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 217/415] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:17 2022] GET /identity/v3/projects/20988f0d51544772b78672526cf109cb => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:17.648153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.754679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb9c5d-1c23-4da9-a65c-d5a5b3a8d036 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.769467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606fa20-9138-411c-9f44-7dd4d9eb3d0d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:17.786289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:17.800684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.801799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.802309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.802781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/groups/633680b351a2424dbde8731f7438ea18/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.805569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] RBAC: Authorizing `identity:create_grant(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13, group_id=633680b351a2424dbde8731f7438ea18, role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:17.807345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:17.808957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:17.809607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:17.810140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:17.810412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:17.813811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:17.824184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:17.825429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 199/416] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 14:18:17 2022] PUT /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13/groups/633680b351a2424dbde8731f7438ea18/roles/5a02bc464efd499bb3230455492613ae => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:17.826469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:17.835315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:17.836299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:17.837078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:17.841742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 218/417] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:17.842773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:17.999665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6782386-1129-4265-97ad-441a8c5e550f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:18.044602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:18.045144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.045425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.045898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.047020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:18.047368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:18.049314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:18.053652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ebb94-e532-4862-bbc3-f6b1fd529bfc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:18.060613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 219/418] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:18.067132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:18.095964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:18.096640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.096977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.097227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] PATH_INFO: `/v3/projects/20988f0d51544772b78672526cf109cb/groups/71855f429d984031a65d64e0d9f75dbf/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.099444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] RBAC: Authorizing `identity:create_grant(project_id=20988f0d51544772b78672526cf109cb, group_id=71855f429d984031a65d64e0d9f75dbf, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:18.101011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:18.117726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 200/419] 10.222.0.50 () {58 vars in 1442 bytes} [Mon Aug 8 14:18:17 2022] PUT /identity/v3/projects/20988f0d51544772b78672526cf109cb/groups/71855f429d984031a65d64e0d9f75dbf/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:18.139734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:18.192645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202df0f-bccf-4779-a392-5b3171702a1d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:18.232687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4af1c-41b0-4b3d-b212-6b4fb367cfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:18.233308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4af1c-41b0-4b3d-b212-6b4fb367cfae None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.233798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4af1c-41b0-4b3d-b212-6b4fb367cfae None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.234278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4af1c-41b0-4b3d-b212-6b4fb367cfae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.323310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4af1c-41b0-4b3d-b212-6b4fb367cfae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:18.324807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4af1c-41b0-4b3d-b212-6b4fb367cfae None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:18.406077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:18.460889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 220/420] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:18.466128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:18.524473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:18.594474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee362900-e612-4ccf-8309-d2fe74e2d5a8 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.594776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee362900-e612-4ccf-8309-d2fe74e2d5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.594937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee362900-e612-4ccf-8309-d2fe74e2d5a8 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.596441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 221/421] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:18.601804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.602371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.602837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.606678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e120ce5f-a273-4178-a49e-ed195aaf64bf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:18.623170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:18.624816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:18.661843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:18.662491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.662773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.663050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.671061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:18.672597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:18.675185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 201/422] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:18 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:18.680654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:18.736271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 222/423] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:18.801017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:18.868726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6603f-6c62-4413-8b79-af381d17aa26 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:18.896285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:18.956813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:18.957892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:18.958377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:18.958840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:18.965106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:18.967781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:18.973855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 202/424] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:18 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:18.979059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:19.037570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:19.056198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da708ae-f76a-408f-812e-1ed074f81da0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:19.133430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0535b-a4b4-461f-9bcb-101fc6fe19c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:19.133993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0535b-a4b4-461f-9bcb-101fc6fe19c0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.134272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0535b-a4b4-461f-9bcb-101fc6fe19c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.134543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0535b-a4b4-461f-9bcb-101fc6fe19c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.159544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc0535b-a4b4-461f-9bcb-101fc6fe19c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-dfc0535b-a4b4-461f-9bcb-101fc6fe19c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:19.165094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:19.166000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:19.166785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:19.169784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 223/425] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:19.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c387c-4a00-4307-b223-34e0fc4f87ad None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.172487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c387c-4a00-4307-b223-34e0fc4f87ad None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.173025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c387c-4a00-4307-b223-34e0fc4f87ad None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.174702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 224/426] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:19.179184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c387c-4a00-4307-b223-34e0fc4f87ad None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:19.183112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3c19c-50bb-485c-817f-03cc468c4b62 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:19.221832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:19.222558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.222918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.223301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.224139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=e319c2beee9242a98bbbec05ac9e6717) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:19.224535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:19.226265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aa456-0c44-4eb8-a1bd-a498df4da3b0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:19.234218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 203/427] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:18 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:19.241108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.241500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.241807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.286574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:19.287855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:19.345210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c387c-4a00-4307-b223-34e0fc4f87ad None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:19.377962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:19.422809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22979552-d49d-490c-b774-433b67367c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:19.423434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22979552-d49d-490c-b774-433b67367c50 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.424061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22979552-d49d-490c-b774-433b67367c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.424350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22979552-d49d-490c-b774-433b67367c50 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.425430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22979552-d49d-490c-b774-433b67367c50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:19.425782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22979552-d49d-490c-b774-433b67367c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:19.427429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22979552-d49d-490c-b774-433b67367c50 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:19.430919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 204/428] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:19.437194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 225/429] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:19.445728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:19.463734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22979552-d49d-490c-b774-433b67367c50 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:19.482363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:19.587358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8a0b1-4163-4fa0-bd44-280ed8f71e04 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:19.597098 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22979552-d49d-490c-b774-433b67367c50 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:19.634169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:19.634615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.634901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.635201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.646869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:19.647683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.648031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.648318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.661636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:19.669093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:19.669993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 14:18:19.670767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:19.676028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 226/430] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:19.679029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:19.719556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:19.741123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:19.743796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:19.780584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:19.834470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4379-2836-4e85-84a6-ce7943b89a1a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:19.897306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 205/431] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:19.902204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:19.902833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:19.903138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:19.903418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:19.904934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:19.911418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:19.912108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:19.913722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:19.924642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 227/432] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:18:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:19.931168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.010119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.053469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4cf90-45a4-42c4-962e-5b11452b40df None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.088049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb9e42-01e8-4b29-9db3-c682ee6739e1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.111597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.111805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.112080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.112361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] PATH_INFO: `/v3/projects/e319c2beee9242a98bbbec05ac9e6717` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.115190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] RBAC: Authorizing `identity:get_project(project_id=e319c2beee9242a98bbbec05ac9e6717)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.116847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.118502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 206/433] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:19 2022] GET /identity/v3/projects/e319c2beee9242a98bbbec05ac9e6717 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:20.127167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.140687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.150940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c4ab-6f70-4cb8-be3b-acb7bb534571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.151866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c4ab-6f70-4cb8-be3b-acb7bb534571 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.152152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c4ab-6f70-4cb8-be3b-acb7bb534571 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.152428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c4ab-6f70-4cb8-be3b-acb7bb534571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.263573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803e4ac-c532-4226-9b84-a8b13a2c5c5a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.263964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c4ab-6f70-4cb8-be3b-acb7bb534571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.265813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c4ab-6f70-4cb8-be3b-acb7bb534571 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.309319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.310413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.310698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.310976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.313052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.314595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.316970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 207/434] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:20.322644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.338085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.404315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 228/435] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:18:20.405549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf618-bcb1-47bb-8f19-5395e41e0143 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.405840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf618-bcb1-47bb-8f19-5395e41e0143 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.406112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5caf618-bcb1-47bb-8f19-5395e41e0143 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.407536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 229/436] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:20.412007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf618-bcb1-47bb-8f19-5395e41e0143 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.446470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.463028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0306c1-e5e4-4b21-9286-1f72dd4f1640 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.498365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.498980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.499944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.500234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.501040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=e319c2beee9242a98bbbec05ac9e6717) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:20.501352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.502791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e7a60-bc41-4fe8-9b80-8d01b3e83d87 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.515598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 208/437] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=e319c2beee9242a98bbbec05ac9e6717&role.id=5a02bc464efd499bb3230455492613ae => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:20.515598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a5342-4a1f-42c6-8295-23d1d6c770ed None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.515598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a5342-4a1f-42c6-8295-23d1d6c770ed None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.515598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a5342-4a1f-42c6-8295-23d1d6c770ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.523985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5caf618-bcb1-47bb-8f19-5395e41e0143 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.538140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-af2a5342-4a1f-42c6-8295-23d1d6c770ed None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:20.539866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-af2a5342-4a1f-42c6-8295-23d1d6c770ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:20.566308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.566960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.567270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.567550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.568714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.570209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.572310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 230/438] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:20.578134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.628295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 209/439] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:20.629683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ba0b5-b7de-47e1-86cc-8783f1daa3fd None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.629972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ba0b5-b7de-47e1-86cc-8783f1daa3fd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.630247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ba0b5-b7de-47e1-86cc-8783f1daa3fd None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.631883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 210/440] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:20.637649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.637934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.638194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.658476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.664364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:20.667001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:20.702409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.706688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c77e8-a341-4eef-9eb7-b8bc99ff0ba9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.749593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.750219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.750503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.750780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.751989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.753455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.755408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 231/441] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:20.756239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.765244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 211/442] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:20.768825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.893821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca370b6-3dc9-461d-b132-e9414a9e145b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.924426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc180ea-27a1-4511-b5df-be762f25d529 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:20.943776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.944429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.944713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.944986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.956812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:20.961745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:20.963623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:20.964584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:20.965322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:20.965322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:20.965322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:20.965322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:20.965627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=575750f3d1c5499bbbb22af90adfb176, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:20.965627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:20.966642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:20.967246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 232/443] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:20.968849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:20.969375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:20.974741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 212/444] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=575750f3d1c5499bbbb22af90adfb176 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:20.975578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.004991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.107422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985b310-f0dd-42a5-a959-77ae2d188896 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.124261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81811ce2-9348-4aba-9c4f-2f3775d20688 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.152640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.153286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.153559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.153825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.167247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:21.169641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.169793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.169965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.170242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.170958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:21.171300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:21.172079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:21.172928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:21.173856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:21.174468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.174535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 233/445] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:21.179119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.183688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 213/446] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:21.190102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.270269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.295868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.313202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f95d-0e43-4250-bb32-890ebd039d0f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.329531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30101e2-1ac4-404c-a4fd-a60b18d4ba9d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.362854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.363621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.364012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.364187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.365375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:21.365479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.366932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.378735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 234/447] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:21.383156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.384014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.384292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.384373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.384564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.459572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.461047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.492678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39080e71-0319-42d3-bf4b-e19e64e28fe9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.520702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.529514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.530075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.530845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.531136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.551486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.567158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 214/448] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:18:21.572822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.615765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.617143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.680639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c94ea-b947-486f-b6b4-5f19dbd749b2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.716382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.717009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.717285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.717564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] PATH_INFO: `/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.719258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] RBAC: Authorizing `identity:get_project(project_id=aeb1cf9e9e9b4eb3b60506cb49252b13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.720730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.721557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 235/449] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:18:21.723596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 215/450] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/projects/aeb1cf9e9e9b4eb3b60506cb49252b13 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:21.727151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.728834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.758335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.777980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.831070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11d82d-b479-4a6d-9f42-fb9383a306ab None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.837748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb74597-2714-4500-aa35-3b97e7ffb673 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.867380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.867931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.868217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.868494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] PATH_INFO: `/v3/projects/20988f0d51544772b78672526cf109cb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.869636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] RBAC: Authorizing `identity:get_project(project_id=20988f0d51544772b78672526cf109cb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.871110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.873211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:21.873933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:21.874244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:21.874516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] PATH_INFO: `/v3/groups/633680b351a2424dbde8731f7438ea18` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:21.875249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 236/451] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/projects/20988f0d51544772b78672526cf109cb => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:21.875725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] RBAC: Authorizing `identity:get_group(group_id=633680b351a2424dbde8731f7438ea18)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:21.877818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:21.878353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.884290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 216/452] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/groups/633680b351a2424dbde8731f7438ea18 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:21.885297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:21.988192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:21.995624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a848246-e021-483a-9c66-654d4b14a2cc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:21.998752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed45936-5f03-4993-8fe0-3e641a9e4d81 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:22.018774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:22.032272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:22.032893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:22.033175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:22.033461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:22.034186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] RBAC: Adding query filter params (group.id=633680b351a2424dbde8731f7438ea18, role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:22.034496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:22.035960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:22.037436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf8e82-be1e-43ca-9312-52f4e5f3e55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:22.038388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf8e82-be1e-43ca-9312-52f4e5f3e55e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:22.038667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf8e82-be1e-43ca-9312-52f4e5f3e55e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:22.038943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf8e82-be1e-43ca-9312-52f4e5f3e55e None admin] PATH_INFO: `/v3/groups/71855f429d984031a65d64e0d9f75dbf` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:22.040535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf8e82-be1e-43ca-9312-52f4e5f3e55e None admin] RBAC: Authorizing `identity:get_group(group_id=71855f429d984031a65d64e0d9f75dbf)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:22.042242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf8e82-be1e-43ca-9312-52f4e5f3e55e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:22.042363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 217/453] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/role_assignments?group.id=633680b351a2424dbde8731f7438ea18&scope.project.id=aeb1cf9e9e9b4eb3b60506cb49252b13&role.id=5a02bc464efd499bb3230455492613ae => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:22.043851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 237/454] 10.222.0.50 () {56 vars in 1180 bytes} [Mon Aug 8 14:18:21 2022] GET /identity/v3/groups/71855f429d984031a65d64e0d9f75dbf => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:22.053690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:22.190541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0586c-ff06-4c13-b8cd-3f5444c90eee None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:22.239613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:22.240223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:22.240504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:22.240775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:22.241509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] RBAC: Adding query filter params (group.id=71855f429d984031a65d64e0d9f75dbf, role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=20988f0d51544772b78672526cf109cb) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:22.241822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:22.243285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0671fc8-755c-494f-8f88-582c6cda2a8b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:22.249945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 218/455] 10.222.0.50 () {56 vars in 1376 bytes} [Mon Aug 8 14:18:22 2022] GET /identity/v3/role_assignments?group.id=71855f429d984031a65d64e0d9f75dbf&scope.project.id=20988f0d51544772b78672526cf109cb&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:24.629746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65137c18-165d-4eab-8bd8-755b25bde9c9 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:24.630266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65137c18-165d-4eab-8bd8-755b25bde9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:24.630341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65137c18-165d-4eab-8bd8-755b25bde9c9 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:24.631932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 238/456] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:24.637187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b58e7-7294-4fbc-bc7f-969b542d8c14 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:24.637964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b58e7-7294-4fbc-bc7f-969b542d8c14 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:24.638411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b58e7-7294-4fbc-bc7f-969b542d8c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:24.664695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-045b58e7-7294-4fbc-bc7f-969b542d8c14 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:24.666120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-045b58e7-7294-4fbc-bc7f-969b542d8c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:24.692254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:24.761736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 219/457] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:24.794100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44ffa6-eec0-4627-8300-c8fa74aebf2f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:24.794860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44ffa6-eec0-4627-8300-c8fa74aebf2f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:24.795364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44ffa6-eec0-4627-8300-c8fa74aebf2f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:24.797158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 239/458] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:24.802316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44ffa6-eec0-4627-8300-c8fa74aebf2f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:24.802585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866f2db-a6ca-40d3-bd52-ddef5d922cac None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:24.802877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866f2db-a6ca-40d3-bd52-ddef5d922cac None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:24.803193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866f2db-a6ca-40d3-bd52-ddef5d922cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:24.829478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b866f2db-a6ca-40d3-bd52-ddef5d922cac None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:24.830698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b866f2db-a6ca-40d3-bd52-ddef5d922cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:24.915400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 220/459] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:24.947361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ce84b-22de-41ec-a1b7-b95559b95b5a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:24.947878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ce84b-22de-41ec-a1b7-b95559b95b5a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:24.948393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ce84b-22de-41ec-a1b7-b95559b95b5a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:24.953135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 221/460] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:24.959175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:24.959462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:24.959742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:24.961570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44ffa6-eec0-4627-8300-c8fa74aebf2f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:24.982706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:24.984490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:25.001740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.002722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.003167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.003761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.016278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.018110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:25.036183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:25.037155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:25.037990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:25.037990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:25.037990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:25.040203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 240/461] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:25.041251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.085636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 222/462] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:25.086680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.087974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.119587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.160735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d248a-c0c8-4285-9ab4-ed86c535a55e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.197987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.198901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.199388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.200109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.212022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.214050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.217007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8ce09-0064-4889-9f75-58a5e06bcb63 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:25.230433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:25.231396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:25.232406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:25.232406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:25.232406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:25.234819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 241/463] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:25.235888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.261192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.261948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.262232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.262504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.263353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:25.263581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.265067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.284562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 223/464] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:25.285636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.334925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.346032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c782dc-f5ca-4c3c-b4b1-a6927d7cf6fc None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.370615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.382922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.383841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.384270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.384690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.396411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.398032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1caf4-6350-4b8f-b4bd-ccfb945f8b78 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:25.413532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:25.414601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:25.415644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:25.415644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:25.415644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:25.417316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 242/465] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:25.418201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.447581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.448508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.448806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.449195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.449918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:25.450299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.452596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.471449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 224/466] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:25.472629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.526445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0618572-2b47-412a-8bb0-f5bae281b157 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.562373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.563286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.563790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.564199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.565068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.566643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.585560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.587202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99dda50-3923-42a4-b943-97feeceae3f1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.593405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 243/467] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:18:25 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:25.594644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.659815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.660488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.660716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.660991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.661758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:25.662071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.663612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.665417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.682741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 225/468] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:25.692269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:25.717068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f080bb-1429-4d3a-8683-cfbaaba45daf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.758281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757469a-594d-475a-8263-d41df0cada15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.759375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757469a-594d-475a-8263-d41df0cada15 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.759872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757469a-594d-475a-8263-d41df0cada15 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.760311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757469a-594d-475a-8263-d41df0cada15 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.761148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757469a-594d-475a-8263-d41df0cada15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.762814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757469a-594d-475a-8263-d41df0cada15 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.790871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 244/469] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:18:25 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:25.835635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57559f-1c8f-4201-8d55-cae83f98a824 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.836201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57559f-1c8f-4201-8d55-cae83f98a824 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.836691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57559f-1c8f-4201-8d55-cae83f98a824 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.838630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 245/470] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:25.845093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.845700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.846205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.879848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:25.882456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:25.884975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1d19f-24e9-4d24-b312-51c445295bc2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:25.949746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:25.951126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:25.951772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:25.952550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:25.954871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:25.957825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:25.963513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:25.984148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 246/471] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:25.997632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 226/472] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:18:25 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:26.016091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:26.143270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c36bf-539e-46ed-8782-27213c2e3d5e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:26.184131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:26.185194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.185566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.185953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.199401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:26.201008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-68c46ba3-ac24-4ab4-8a66-ed762773a344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:26.218808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:26.219945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:26.220948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:26.220948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:26.220948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.222417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 247/473] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:26.229791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:26.252937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda4204-aaa0-4fdf-8f23-b457ee85e8a3 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.253383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda4204-aaa0-4fdf-8f23-b457ee85e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.253756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda4204-aaa0-4fdf-8f23-b457ee85e8a3 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.256263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 248/474] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:26.261504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.261785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.262057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.282324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:26.283587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:26.306171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:26.327940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:26.372299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd780285-3d1c-4c68-b91b-4ccb9ad400a7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:26.381060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 249/475] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:26.404608 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:26.417222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:26.418105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.423114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.423765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.424732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:26.425182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:26.426710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:26.453268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 227/476] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:26.461329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:26.573613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf2233-5e50-41c1-b214-b3901a6d4b57 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:26.611463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:26.612178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.612530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.612876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.613632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:26.614383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5df14-00fe-47ac-8474-d808b50cb383 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:26.615230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:26.627678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.sql.core [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 14:18:26.627678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 14:18:26.627678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [parameters: {'id': 'cda1ebe232b14bccae8c14e3a3c3577c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 14:18:26.627678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80134) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 14:18:26.637432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application conn.query(q) Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result.read() Aug 08 14:18:26.653915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 14:18:26.655552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application next(self.gen) Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application yield resource Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application next(self.gen) Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise self.value Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application session.commit() Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 14:18:26.656666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.session.flush() Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application compat.raise_( Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 14:18:26.657677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 14:18:26.659159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application conn.query(q) Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result.read() Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 14:18:26.659991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [parameters: {'id': 'cda1ebe232b14bccae8c14e3a3c3577c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:26.660761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 14:18:26.661539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.662395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 250/477] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 14:18:26 2022] POST /identity/v3/roles => generated 144 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:26.672198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:26.673456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:26.674303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.674662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.675049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.675862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:26.677400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:26.707251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 228/478] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:18:26 2022] POST /identity/v3/users => generated 249 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:26.759479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:26.809058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6f776-34a7-454a-b157-937eb1009e21 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:26.849582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f66b851-aba3-4e8a-80d2-716dec8d8c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:26.850251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66b851-aba3-4e8a-80d2-716dec8d8c57 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:26.850605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66b851-aba3-4e8a-80d2-716dec8d8c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:26.850949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66b851-aba3-4e8a-80d2-716dec8d8c57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:26.867116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f66b851-aba3-4e8a-80d2-716dec8d8c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-0f66b851-aba3-4e8a-80d2-716dec8d8c57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:26.869094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:26.870133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 14:18:26.871107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:26.872723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 251/479] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:18:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:26.879017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:26.999490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14decd47-7f49-486c-99f1-02c0ba6d0dea None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:27.036398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:27.037205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:27.037566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:27.037912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:27.038706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:27.039106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:27.040687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782758f3-65fd-421e-a0cd-f38595d9724e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:27.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 229/480] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:28.357878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedb798-58f4-4275-a535-64b1b17cd20e None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.358954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedb798-58f4-4275-a535-64b1b17cd20e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.358954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedb798-58f4-4275-a535-64b1b17cd20e None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.360664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 252/481] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:28.366300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.366761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.366846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.392104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:28.393304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:28.425841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:28.464687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85c104-e4dc-4a9d-a093-534aa7a9460c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.464977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85c104-e4dc-4a9d-a093-534aa7a9460c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.465221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b85c104-e4dc-4a9d-a093-534aa7a9460c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.466530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 253/482] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:28.473782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1cc9b-2565-4bae-8662-65ab8f758a9b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.474052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1cc9b-2565-4bae-8662-65ab8f758a9b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.474314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e1cc9b-2565-4bae-8662-65ab8f758a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.497338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-94e1cc9b-2565-4bae-8662-65ab8f758a9b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:28.498523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-94e1cc9b-2565-4bae-8662-65ab8f758a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:28.505564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 230/483] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:28.567388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:28.602466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 254/484] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:28.605232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a784674-1912-4a76-887b-7b2a02f0ee6b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.605735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a784674-1912-4a76-887b-7b2a02f0ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.606228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a784674-1912-4a76-887b-7b2a02f0ee6b None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.609580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 255/485] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:28.619072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.619602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.620089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.642477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:28.643852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:28.660062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:28.723812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 256/486] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:28.725210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:28.731754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01532074-8ea6-4960-96a6-045cf7123a16 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:28.776132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:28.776911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.777201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.777470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.791898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:28.798698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:28.799641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:28.800420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 231/487] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:28.801074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:28.826264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:28.854292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f59a-5ba4-4f76-8118-dd71bf2cb529 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:28.897057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:28.897955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.898243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.898515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.911759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:28.917609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:28.918572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:28.919402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 257/488] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:28.921144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:28.924306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0b249-45ba-470f-bed7-261aa63515b7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:28.934839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:28.964327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:28.965387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:28.965864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:28.966331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:28.979017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:28.985065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:28.985948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:28.986729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:28.989188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 232/489] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:28.990180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.041919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d97b8-d1b0-4ea2-b5ea-6f4f8128a983 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.080668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.081705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.082240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.082774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.083686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:29.084225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.085906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.095005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.096193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 258/490] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:29.097248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.129514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433b071-ef50-41c7-bb96-2661a54d4109 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.167637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.168584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.169063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.169551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.170434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:29.170956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.172581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.181689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 233/491] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:29.182884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.190922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.232545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa3e92-b9a6-4424-b099-3cc72c64b57b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.275382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.276388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.276894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.277361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.278238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:29.278747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.281077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.291390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 259/492] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:29.292778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.313000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5729ec-5449-4e97-93b0-6f55fe132caa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.362079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.362771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.363123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.363804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.377127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.379459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.381423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:29.400054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:29.400985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:29.401835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:29.401835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:29.401835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:29.403915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 234/493] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:29.404959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.424052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a93f8-3537-4f1a-ad2e-3487ad8965c1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.462373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.463069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.463374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.463644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.466406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.479380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.480856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:29.495532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:29.496451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:29.497316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:29.497316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:29.497316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:29.497316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 260/494] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:29.497542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.539147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc2c7c-2fd7-41fe-ae12-eccc19bda2fa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.580300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.581567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.581813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.582307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.604496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.606012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.612745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fa364-11d3-431e-be1f-4521eccd9531 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8712ff37-c735-4a17-bc19-029249bf77a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:29.622746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:29.623823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:29.624733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:29.624733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:29.624733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:29.624733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 235/495] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:29.625754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1641-cb78-4ce1-8b2b-f026fde55f1f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.625754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1641-cb78-4ce1-8b2b-f026fde55f1f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.625754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c1641-cb78-4ce1-8b2b-f026fde55f1f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.627288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 236/496] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:29.629300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c1641-cb78-4ce1-8b2b-f026fde55f1f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.665993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.666613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.666896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.667200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.667857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:29.668164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.669558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.681751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.682389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 261/497] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:29.683391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.735508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.744593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c1641-cb78-4ce1-8b2b-f026fde55f1f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.781120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.781738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.782022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.782299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.782957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:29.783306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.784711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f62c7-c9b7-4bbf-9126-7de2370b29ff None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.796531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858eae-7625-4bdf-acd0-3572689d1a8b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.798294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 237/498] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:29.799473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.799757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.800026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.819070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:29.820314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:29.838551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:29.839507 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.839786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.840057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.840767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:29.841074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:29.842972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be44ad-378b-4ac3-902b-46b3094428b6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:29.855208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 262/499] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:29.856469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e8f78-eda0-41be-9439-61632b86d0ee None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:29.856754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e8f78-eda0-41be-9439-61632b86d0ee None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:29.860515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e8f78-eda0-41be-9439-61632b86d0ee None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:29.861938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 263/500] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:29.862681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e8f78-eda0-41be-9439-61632b86d0ee None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.895112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 238/501] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:29.895965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:29.940064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:29.978123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e8f78-eda0-41be-9439-61632b86d0ee None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:29.984960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.006454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d21d1b-8c83-4464-8089-1bbfec6bb670 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.011746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.012382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.012664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.012937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.022244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.023966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:30.037304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:30.038331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 264/502] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:30.039225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.048144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.048800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.049081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.049353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.056651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.058101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2fc05f8b-6c12-4790-bac3-25710f5eb533 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:30.071181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:30.072288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:30.073173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 239/503] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:30.074169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.074169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.074169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.093087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:30.094399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:30.147818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d568c-2362-4c77-92ec-4f97c36668ed None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.163556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 240/504] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:30.164392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.189351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.197340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.197914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.198190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.198460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.206816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.208289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:30.219483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:30.220321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:30.221194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 265/505] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:30.221731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.249161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.272924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cec773-9462-42a5-b9da-6f8843f62f81 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.306555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.307208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.307488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.307764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.318435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:30.322593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68d96b-156c-466a-ada3-feaaa3ce5e7d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:30.323048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:30.323950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:30.324722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 241/506] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:30.325354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.359800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.360405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.360683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.360951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.361676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:30.361979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.363427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.400206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 266/507] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:30.401038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.408983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.425335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526402-bba8-45e4-a997-389f06cb0d1f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.461205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33027089-58de-43fe-970c-1e931d580feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.461754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33027089-58de-43fe-970c-1e931d580feb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.462040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33027089-58de-43fe-970c-1e931d580feb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.462310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33027089-58de-43fe-970c-1e931d580feb None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.463031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33027089-58de-43fe-970c-1e931d580feb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:30.463360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33027089-58de-43fe-970c-1e931d580feb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.464786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33027089-58de-43fe-970c-1e931d580feb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.486141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.502403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742d1d4-45ff-4e97-8f21-1089ae257977 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.502595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 242/508] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:30.503270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33027089-58de-43fe-970c-1e931d580feb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.538617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.539276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.539557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.539830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.550494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.551992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:30.566475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:30.567425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:30.568247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:30.568247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:30.568247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:30.568247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 267/509] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:30.568453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.605259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33027089-58de-43fe-970c-1e931d580feb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.640079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.640684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.640962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.641251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.641951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:30.642253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.643733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.658327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.662606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 243/510] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:30.664010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.675362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad558a40-f1e3-452c-b1e6-6ed9d0138519 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.710035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.710649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.710926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.711292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.711903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:30.712207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.713590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.715846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.722966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 268/511] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:30.723797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:30.774160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c38a9-22a2-40a9-917b-bc2ed94ae6c7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.814945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.815577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.815859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.816136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.827770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6213441e-e5e1-40f9-99af-4b0d1b1e493c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:30.862664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:30.863306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:30.863603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:30.863859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:30.881437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.884250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.886010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.928971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:30.932691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:30.934727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:30.988711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 244/512] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:30.989518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.039308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 269/513] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:31.040227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.105557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.109430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7309e21-8720-455d-ae51-5e5036cc0d9d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.146814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.147456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.147732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.148006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.148645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:31.148943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.150332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.156574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.160454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d291b3-ceff-4fb1-bba7-a542f4645a89 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.162853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 245/514] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:31.163707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.201696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.202419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.202699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.202968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.277715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32e4ad-447d-4a92-9069-003f9e1dc0b5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.285384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.286868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.318524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.319321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.319753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.320153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.332283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.333756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:31.348717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:31.349701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:31.350532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:31.350532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.350532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:31.350532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 246/515] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:31.350739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.366801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.392416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.399980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 270/516] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:31.400786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.459318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fed6e-4b97-4eec-ae16-3425316910b3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.494289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.494935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.495247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.495527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.503919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2847968-b570-4f35-8ff6-c4fe6894777d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.506542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.508032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:31.522369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:31.523334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:31.524210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:31.524210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.524210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:31.527390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 247/517] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:31.528410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.539461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.540092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.540370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.540636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.550481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.551957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:31.566417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:31.567372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:31.568176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:31.568176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.568176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:31.568176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 271/518] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:31.568393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.591695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.617527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.631114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efb0a4-cec6-45d8-8338-7a165b4f2ac6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.666305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.666906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.667218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.667491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.667822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ab8a2-ad09-4fa5-a76d-d000188b49b8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.668123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.669541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.691691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 248/519] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:18:31 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:31.692809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.702336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.702967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.703344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.703639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.704485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:31.705645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.706230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.720587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 272/520] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:31.721367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.823432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a187172-7c39-4066-99aa-dcb6f25af6dd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.841662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0671edd-32b4-4619-b50a-515f34110874 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:31.853608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.856709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:31.871836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.872875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.873161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.874564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.885593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.885949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:31.886565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:31.886843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:31.887064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.887209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:31.888065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:31.888165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:31.894346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:31.904954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 273/521] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:31.905798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:31.909282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:31.910296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:31.911149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:31.911149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:31.911149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:31.911149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 249/522] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:31.911970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.018358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d35dc6-4043-4c78-aa7a-f17706deb2b7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.022224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20556bd4-c74e-4222-a2e1-da52e9df0eac None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.057040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.057677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.057937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.058216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.058868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:32.059459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.060125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.061369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.061669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.062138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.062626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.071539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.073264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.073862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 274/523] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:32.074657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:32.090199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:32.091318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 250/524] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:32.092461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.118304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.130977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.211627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499797d7-1c67-46f8-9c88-9eb476b5aa8b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.219746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32b4a7-a32f-4929-8c15-31eecd9c85a7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.249813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.250450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.250723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.250996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.258914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.260409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.261338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.262208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.262487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.262782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.263546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:32.263856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.265293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:32.272688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:32.273561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 275/525] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:32.275485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.288102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 251/526] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:32.289044 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.373433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.386206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd37268-4f23-40b4-b158-bf8701f545c2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.396653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.408527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceb558-6a66-43ec-aa45-7d5efc59da1a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.426858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.427528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.427808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.428179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.436110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.437697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.446862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.447579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.447934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.448281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:32.448986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:32.449841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:32.450731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:32.451171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:32.451171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.451171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.455504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 276/527] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:32.456027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.472020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 252/528] 10.222.0.50 () {56 vars in 1139 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:32.472872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.574513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c9e855-1728-4b66-972f-817a0d14377f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.586700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6ac5d-a9ae-4bc8-b50b-b3f3da3e85be None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.615522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.616150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.616538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.616761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.617806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:32.618120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.620311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.628561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.629138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.629421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.629695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.636811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.638309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.647708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 277/529] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:32.647708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:32.655207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:32.656750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 253/530] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:32.657608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.664956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.674021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.762589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c58af9-9a0d-4185-8698-d56a402fe199 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.773990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8e804-e5c4-426e-8c3f-ba4910b3534e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.799952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.800545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.800828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.801103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.801808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:32.802117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.803655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.813279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.814133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.814133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.814429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.817265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 278/531] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:32.818095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:32.901238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.902709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.905089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.914412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:32.943155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a764ec2-0b2a-484a-b36d-8a6454366422 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:32.979882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:32.980527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:32.980775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:32.981048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:32.981782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=a587bc4f9082494ab0ab2b6ccbd517ee) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:32.982097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:32.984237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:32.999125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 279/532] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/role_assignments?user.id=a587bc4f9082494ab0ab2b6ccbd517ee&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:32.999125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:33.025094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 254/533] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:33.026110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:33.141553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fecd27-f65f-44e7-a9e3-175234f059dd None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:33.147390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52defef-926c-4c50-b5b9-e604830b0f0f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:33.160719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:33.193779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:33.194204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:33.194397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.194693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.194969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.195565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.195783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=9355f8f80a434ab8af3145f7e82c277c) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:33.196093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:33.196386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.196823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.197535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:33.197867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:33.198338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:33.200302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:33.203314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:33.211114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 255/534] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:33 2022] GET /identity/v3/role_assignments?user.id=9355f8f80a434ab8af3145f7e82c277c&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:33.211114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:33.233648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 280/535] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:33.239909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:33.422488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a930dc5-dcd8-4dc5-a9a6-e6dda4f1b094 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:33.444520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186f256-dbe9-4142-95cd-92e90a2ca352 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:33.488974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:33.489649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.489886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.490155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.496790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:33.497704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.498180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.498698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.499770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=d22e2d936518407fba2d0be379598ce6) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:33.500279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:33.502268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:33.509427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 281/536] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:33 2022] GET /identity/v3/role_assignments?user.id=d22e2d936518407fba2d0be379598ce6&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:33.520534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:33.522017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:33.538591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:33.539596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:33.540403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:33.540403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:33.540403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:33.541017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 256/537] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:33.546726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:33.649648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:33.704886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afa650-e75d-40be-baa0-892ebd9c90b5 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:33.740261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:33.740878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.741160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.741436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.742084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:33.742389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:33.743811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa736f-31ce-4132-9ba5-64c808b20cd0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:33.757044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 282/538] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:33.762162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:33.815386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:33.872459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715f64-b415-40b7-aae0-00a8ffcbfa1a None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.872780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715f64-b415-40b7-aae0-00a8ffcbfa1a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.873019 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89715f64-b415-40b7-aae0-00a8ffcbfa1a None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.876266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 283/539] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:33.881110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.881398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.881678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:33.919509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:33.922876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:33.956179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b17115-c2c4-4229-8511-9a89ea13338d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:33.994704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:33.995591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:33.995872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:33.996143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.012409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:34.013965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:34.035864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:34.036734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:34.037549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:34.043111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 257/540] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:34.044640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:34.057744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 284/541] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:34.103932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:34.149157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:34.169749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd40ef5-94f8-452d-a911-a27e36c1b5a3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:34.230394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:34.231042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:34.231974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:34.232342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:34.232609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.233519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:34.234687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:34.236279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:34.256505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 258/542] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:34.262274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:34.333082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c307b-b2e7-4cf1-b594-95f64476f411 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:34.372498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:34.373138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:34.374047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:34.374385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.386622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:34.391686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:34.392542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:34.393323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 285/543] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:34.404490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:34.488198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8826f0-6a0b-4286-bab8-8362e3b57842 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:34.551127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:34.551127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:34.551127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:34.551127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.551961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=511ecf1b54294d088d4737c1841cad5b) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:34.551961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:34.552192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:34.558825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 259/544] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:34 2022] GET /identity/v3/role_assignments?user.id=511ecf1b54294d088d4737c1841cad5b&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:34.570820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ba0bc-73e5-48dd-bb31-499e9d34012e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:34.581505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:34.614294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:34.615250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:34.615538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:34.615863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.616634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:34.616948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:34.618390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:34.628698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 286/545] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:34.637747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:34.683498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:34.766369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24855aa9-570d-4bad-bf63-2a65b7aa9c94 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:34.811972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:34.812612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:34.812897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:34.813174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.824718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:34.826180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:34.841443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:34.842377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:34.843215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:34.843215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:34.843215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:34.843215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 260/546] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:34.849515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:34.960914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027366e-4486-47ee-b04c-fdd093f1c574 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:34.997006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:34.997632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:34.997915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:34.998714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:34.999405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:34.999714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:35.001113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:35.015129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 287/547] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:35.020534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:35.094333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:35.142617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8852e0-c2de-4716-8a1e-b096876467c5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:35.190194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:35.190836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.191139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.191426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.202935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:35.204446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-822954f6-ca79-43ea-a750-9de0f4cedba3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:35.218589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:35.219585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:35.220929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:35.223313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 261/548] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:35.227969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:35.276534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:35.402569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c96c1-b08e-4f41-a1eb-dcff1be23337 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:35.450134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:35.450748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.451025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.451837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.452579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:35.452879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:35.454377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:35.464837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bcffd-7a30-4904-9615-c27280d19a63 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.465148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bcffd-7a30-4904-9615-c27280d19a63 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.465420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bcffd-7a30-4904-9615-c27280d19a63 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.467951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 262/549] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:35.475164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280074-9e7d-4d12-b429-3ece0aa5a43e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.475486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280074-9e7d-4d12-b429-3ece0aa5a43e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.475824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280074-9e7d-4d12-b429-3ece0aa5a43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.482940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 288/550] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 14:18:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:35.488574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:35.497106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6a280074-9e7d-4d12-b429-3ece0aa5a43e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:35.498877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6a280074-9e7d-4d12-b429-3ece0aa5a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:35.625910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:35.640285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 263/551] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:35.641563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e543-7430-4d7b-91d4-4d8d18bbc55e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.641845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e543-7430-4d7b-91d4-4d8d18bbc55e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.642114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e543-7430-4d7b-91d4-4d8d18bbc55e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.643811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 264/552] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:35.655033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.655505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.655945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.660663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883fc71-de4e-406e-b602-2d756ce09581 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:35.683777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:35.685253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:35.706529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:35.707170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.707457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.707731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.739252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:35.771173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 265/553] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:35.772277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:35.799084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:35.801267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:35.907007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfad030-d1e0-4dc6-ba99-91c207bfdb25 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:35.926761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:35.946137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:35.946855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:35.947228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:35.947567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:35.947719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 289/554] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 14:18:35.948855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:35.961134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:35.965906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:35.966976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:35.967821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:35.971552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 266/555] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:35.972421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.032564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.066228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a0c4-2bde-4c5a-963e-818159849c86 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.098192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae3705-e8c6-465a-912d-5078f684e362 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.100397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.101039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.101321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.101597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.114131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:36.118933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:36.120117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 290/556] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:36.126724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.144078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.144716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.145001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.145273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.156235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.157679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d62aba35-d0cb-4c91-ae4f-0b033c76f94f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:36.176207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:36.177738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:36.178668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:36.178668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:36.178668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:36.178668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 267/557] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:36.178668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81546c-a859-42b3-b04b-a510ddcb0228 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.178668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81546c-a859-42b3-b04b-a510ddcb0228 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.178991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81546c-a859-42b3-b04b-a510ddcb0228 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.179744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.180490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 268/558] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:36.181086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81546c-a859-42b3-b04b-a510ddcb0228 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.242286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29fc66-da0b-49b6-b13e-8b5f744f800b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.279269 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.280539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.280841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.281202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.281873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:36.282179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.283690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.284417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.291826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81546c-a859-42b3-b04b-a510ddcb0228 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.294040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 291/559] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:36.294849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.330682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.331316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.331593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.331953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.332616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:36.332923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.334299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24de122-3788-4572-bda8-a3338aac9e0b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.343300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 269/560] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:36.344466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.344748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.345020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.363542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:36.364713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:36.406303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d586e67-bab7-4024-b0cf-89a701ab1ff3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.416723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.444639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.445242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.445519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.451441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 270/561] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:36.451583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.451583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:36.451762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.451849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.451849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.469996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 292/562] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:36.470963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.553560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.570860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f6a14-8c34-4c67-8b86-515f50061151 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.577635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e068c-1e3f-4438-9a5e-bdf3abd8cad4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.608608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.609232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.609512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.609776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.613231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.613857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.614137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.614409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.620402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.621970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.625159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.626621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:36.637574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:36.638512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:36.641316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:36.641316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:36.641316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:36.641316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 271/563] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:36.642423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:36.643305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:36.645807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:36.645807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:36.645807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:36.646275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.648025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 293/564] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:36.648887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.674976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.755605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe59f7-72cc-4a3b-a006-e377fcbcc3b4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.757196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c59e1-fb8e-4826-b6a0-6b38189ee6f9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.793506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.793721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.794156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.794261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.794436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.794556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.794715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.794818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.802195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.803695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.805823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.806379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:36.812680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:36.813612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:36.814432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:36.816985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:36.817980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:36.819048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:36.820101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 294/565] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:36.820101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.826911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 272/566] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:36.826911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:36.916805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:36.939996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde6fc5-825c-4961-8f12-bd34383bb494 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.942389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df55838-7f42-45e0-a1fe-aeb99dc5ccfe None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:36.976093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.976796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.977157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.977504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.978226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:36.978605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.978699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:36.979348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:36.979691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:36.980026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:36.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.980789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:36.981165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:36.982683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:36.997052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 295/567] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:36.997052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.004615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 273/568] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.004615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.066696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.100238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19dbfc-861b-4789-a21a-47de66945c5b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.115216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cb34-fbde-4889-9dbe-bda45389a2e5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.133802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.134420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.134702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.134973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.135668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:37.135973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.137348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.145578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 296/569] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.146387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.152975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.153126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.153600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.153873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.154145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.155395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:37.155724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.157166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.168895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 274/570] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.169694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.249574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34949512-38e1-4674-ab11-1a33e01a5516 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.268943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075e57d5-2269-41a4-ada0-cf866ec5ffb1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.285458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.286480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.286941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.287451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.290424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.298450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.300139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.310567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.311216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-0042a6a7-7c84-4221-be02-2b24ede075a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:37.311320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:37.312351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:37.313350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:37.313878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.313878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.313998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 297/571] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:37.313998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7d9bd-2b2b-4e1c-9c7f-946ffd2c3e22 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.313998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7d9bd-2b2b-4e1c-9c7f-946ffd2c3e22 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.314177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e7d9bd-2b2b-4e1c-9c7f-946ffd2c3e22 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.315258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 298/572] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:37.315948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7d9bd-2b2b-4e1c-9c7f-946ffd2c3e22 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.325275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.326777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:37.337674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:37.338592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:37.339556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 275/573] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:37.340128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.383808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.424119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e7d9bd-2b2b-4e1c-9c7f-946ffd2c3e22 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.446379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858ebf0-8738-4965-8326-9d9e25ca4592 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.457652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.458269 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.458546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.458818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.469859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.471322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.479684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.480276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.480551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.480819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.481530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=94cf8e7d52a3441196ac4b4a63fb13ec) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:37.481834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.483258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994f97-e1a1-4a32-bd99-8ca7cb13bcef None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:37.484815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:37.485714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:37.486593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:37.486593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:37.486593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:37.486593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 299/574] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:37.487038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.491947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 276/575] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/role_assignments?user.id=94cf8e7d52a3441196ac4b4a63fb13ec&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.493131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.493416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.493692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.521226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:37.522404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:37.563792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.601076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 277/576] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:37.605184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5282-09da-41d4-be7e-d761f9ab6f23 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.613771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.628471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.649475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.650094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.650372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.650673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.651434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:37.651744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.653178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.672540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 300/577] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.673468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.734852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bcda4-16d4-448b-8b15-3f8798c124d1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.771982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.772625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.772910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.773183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.773906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:37.774212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.775910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.778314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868e6ea-3328-415c-ae7a-96766abe649c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.799666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 278/578] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.800837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.821762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.822383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.822649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.822918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.823599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:37.823903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:37.825254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:37.837987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 301/579] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:37.838821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:37.846785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.882578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:37.944134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18ce8-986b-4fe2-bc35-5ee7c29fdb53 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.959933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94220a7c-837f-4298-b522-8e7a8b146935 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:37.983939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.984573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.984908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.985184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:37.996551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:37.997655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:37.998834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:37.999134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:37.999409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:38.002095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:38.003028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:38.003899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:38.007914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 279/580] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:38.008935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.015457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.017022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:38.032112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:38.037938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:38.039038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:38.039038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:38.039038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:38.039038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 302/581] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:38.039038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.118262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.130799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1180c03-cf3e-46f7-ab05-4f796f2ba1d2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.132215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.149133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7336a94-d1c6-48f5-8fbd-f1187092deff None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.168352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.169005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.169263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.170544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.185696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.187707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.191747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.192980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.193468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.193891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.204638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:38.206075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:38.207221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:38.208306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:38.208306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:38.208306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:38.208914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.214094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 280/582] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:38.214094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:38.229204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:38.230279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:18:38.232007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:38.235177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 303/583] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:38.235177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.345218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa843d7-dd58-4331-a58e-f02f8cfe5d6c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.359915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378d80f-7153-4383-b9ea-f774cbc6d5b4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.393430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.394588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.395230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.395553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.396254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:38.396846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.398263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.411855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 281/584] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:38.411855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.417058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.417640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.418288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.418799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.419135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.419757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:38.420590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.422619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.436903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 304/585] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:38.437486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.447662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.556786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb43f7-da5a-45f9-988c-f4ccbafba8c9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.584881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dba04a-083b-488d-a9b5-6d4901a51470 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.594898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.595633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.595854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.596130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.596808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:38.597112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.598569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.617884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 282/586] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:38.618839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.625334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.626068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.626443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.626811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.627672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:38.628185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.629731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.660097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 305/587] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:38.661728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.703131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.731284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.746909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a251-0eb9-497c-a58e-5c4609a19796 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.772518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08bc2a-6ddf-4b68-b6fd-0f0976d2d3b1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.783548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.784283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.784638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.785053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.797568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.799083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.814011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.814729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.815064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.815425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:38.817109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:38.818186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:38.819468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:38.819468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:38.819468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:38.819468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 283/588] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:38.819890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.826127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.828071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:38.841441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:38.843145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:38.844417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 306/589] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:38.845055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:38.944157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bb880-f465-4a2c-b811-b2c69aa6ef45 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.954642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fc3a1-cc51-482a-8944-271c5fccb7bf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:38.962491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.983009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:38.987856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.988665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.988887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.989165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.995168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:38.995796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:38.996074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:38.996338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:38.997287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:38.998763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.007145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.008631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:39.011790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:39.012696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:39.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:39.018105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 284/590] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:39.018750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:39.025386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:39.026283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:39.032547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:39.032547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:39.032547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:39.032547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 307/591] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:39.032547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.137977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317ad6a-436f-4996-8634-5fed23937845 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.152070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc00a2-84d7-4032-81d6-771c569b3d32 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.178192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.178885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.179195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.179473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.180120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:39.180428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.182052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.186526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.187156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.187441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.187707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.188390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:39.188692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.190121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.199399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 285/592] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:39.200600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.204911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 308/593] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:39.209829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.238856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:39.246936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:39.313040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46119a45-ceba-4865-a0d2-4c9614f86a84 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.313934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a09fb-8fee-45e2-8303-d78ec7fdf31c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.349544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.349785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.350158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.350262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.350440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.350549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.350712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.350805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.351393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:39.351489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:39.351703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.351795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.353142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.353243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.369238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 309/594] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:39.369424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 286/595] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:39.369424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.370162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.469652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:39.472162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:39.477257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1e7ec-1d58-49ce-9bab-39540431eb42 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.479830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca29a2-6517-4bdc-b1ba-bdde90ae8768 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.511918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.512551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.512889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.513109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.516323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.516980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.517260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.517532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/a587bc4f9082494ab0ab2b6ccbd517ee/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.521984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.523456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.532055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=a587bc4f9082494ab0ab2b6ccbd517ee, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.533492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:39.534512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:39.535418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:39.536310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 310/596] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:39.537158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.549582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 287/597] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:39 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/a587bc4f9082494ab0ab2b6ccbd517ee/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:39.550121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.659964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6441e47d-a5a7-4540-a29a-c8ca0a41e70d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.663886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71abbeb-0a82-4aa7-a71b-77d55efb1ee3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.700963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.702309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.702578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.702857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.706404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443f9ed-d712-466c-9369-7c9b22642dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.707271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443f9ed-d712-466c-9369-7c9b22642dcd None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.707688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443f9ed-d712-466c-9369-7c9b22642dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.708101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443f9ed-d712-466c-9369-7c9b22642dcd None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/9355f8f80a434ab8af3145f7e82c277c/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.710847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.713565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.725670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443f9ed-d712-466c-9369-7c9b22642dcd None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=9355f8f80a434ab8af3145f7e82c277c, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c7532182-b1e5-43a5-afdb-117da1ccede1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:39.726414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:39.727461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:39.728413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 311/598] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:39.728945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c03ce2-9fca-4ea5-971d-f51d71ef5275 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.729010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c03ce2-9fca-4ea5-971d-f51d71ef5275 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.729248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c03ce2-9fca-4ea5-971d-f51d71ef5275 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.730188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443f9ed-d712-466c-9369-7c9b22642dcd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.730721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 312/599] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:39.731454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c03ce2-9fca-4ea5-971d-f51d71ef5275 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.735657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:39.747552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 288/600] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:39 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/9355f8f80a434ab8af3145f7e82c277c/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:39.750135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.750615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.751127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.755907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:39.770919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:39.772246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:39.859267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 289/601] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:39.860342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:39.890887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c03ce2-9fca-4ea5-971d-f51d71ef5275 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:39.941262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:39.942819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:39.943739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:39.943739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:39.944797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:39.944797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:39.946125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:39.964794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 313/602] 10.222.0.50 () {56 vars in 1139 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:39.965470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.001161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a11878-7212-41f9-baef-f5c3652de69c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.035040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:40.046750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.047504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.047858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.048424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.050133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:40.050523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.052061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.054875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:40.067699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 290/603] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:40.068543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.096804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd383c76-be6c-4014-ae78-bf6e365887d0 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.144946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.146637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.147017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.147415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.165116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:40.171052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:40.171979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:40.172808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 314/604] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:40.173484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.195045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b802e-770c-4851-ba87-4e21e7db3188 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.233256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.233894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.234176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.234453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.249052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.251345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:40.273412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:40.274602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:40.275855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:40.275855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:40.275855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:40.278443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 291/605] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:40.280340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.319124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7d39c-1105-48a9-bb7a-ef1307a3ab95 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.336602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:40.357860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.358675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.358857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.359278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/d22e2d936518407fba2d0be379598ce6/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.361851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:40.376704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=d22e2d936518407fba2d0be379598ce6, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.378214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.395591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 315/606] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:40 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/d22e2d936518407fba2d0be379598ce6/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:40.396540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.407131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7cbe2-cfed-42dd-a022-0358b7b336ec None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.449357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.450061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.450391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.450681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.452068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:40.452383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.454106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.467305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 292/607] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:40.471515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.527996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39df77d-25b1-489a-8ec1-205ee197470d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.575550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.576718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.577005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.577293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.578015 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:40.578323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.580696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.598417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 316/608] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:40.609937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.640839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:40.644281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535fea4-332a-41aa-ae9f-9fd501bac307 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.666349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:40.696302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.696954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.697289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.697563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.712204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.713907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.736970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add720f9-95b2-4d46-b0ed-46588475f0c0 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:40.746980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:40.749219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:40.750317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:40.750317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:40.750317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:40.750317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 293/609] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:40.756008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.779681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:40.780637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:40.781066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:40.781503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:40.789840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:40.791620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:40.804766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:40.805922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:40.806867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:40.809113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 317/610] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:40.814768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:40.912672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a7adc-3bb8-44a6-9e2e-782a9003055b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.977626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b9e2b-a941-43ba-af0b-5649e2bbb427 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:40.980264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:41.003825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.004445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.004726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.004999 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.005648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:41.005950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.007345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.016938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.017628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.017979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.018413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.019197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:41.020197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.021701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.033602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 294/611] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:41.036072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 318/612] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:41.039043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:41.041826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:41.064007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:41.200828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c4486e-657b-4052-a802-3e5cab06997f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:41.232815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc5e76-2837-4c4b-9f8a-28333737451f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:41.267435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.268067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.268338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.268604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.279693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.282282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.298967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.299577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.299852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.300123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/511ecf1b54294d088d4737c1841cad5b/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:41.303463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:41.304562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:41.305693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 295/613] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:41.310489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:41.333079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=511ecf1b54294d088d4737c1841cad5b, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.334813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.354199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 319/614] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:41 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/511ecf1b54294d088d4737c1841cad5b/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:41.432729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:41.469040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb39f5-bb0a-4e2d-a2d7-4ddbf1c69b23 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:41.509576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.510197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.510522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.510795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.511569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:41.511940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.514217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.533248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 296/615] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 14:18:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:41.538360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:41.556786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:41.649553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dac3c-a50a-4d7b-b732-aef53ebb4b2b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:41.685025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.686185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.686457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.686727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.697680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.699229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-90f3290a-365a-464e-a9a3-23f132a619c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:41.722750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:41.723764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:41.724631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:41.724631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:41.724631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:41.724631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 320/616] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:41.729747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:41.845026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed3dd5-7736-4148-8430-8de2dc291cf8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:41.878265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:41.881836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:41.882456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.882740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.883366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.883680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:41.883998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:41.885495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:41.892035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa8919-fcbf-48aa-9d2a-0ca2166922ee None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.892369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa8919-fcbf-48aa-9d2a-0ca2166922ee None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.892584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa8919-fcbf-48aa-9d2a-0ca2166922ee None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.894192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 321/617] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:41.897728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 297/618] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:41.899411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:41.899700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:41.900061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:41.904333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:41.936897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:41.938104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:42.027876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:42.038039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 322/619] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:42.072813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33e215-8baf-4a9c-a9b9-aa538103e794 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.083823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.135555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.136109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.136491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.136761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.148985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:42.150455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:42.173506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:42.174605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:42.175525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 298/620] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:42.182904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.273452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e059d-261e-4aa6-96b5-38bee7c377d8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.282158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:42.324867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ba3cd-7e8b-4522-8d3e-9360c7a33687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.325500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ba3cd-7e8b-4522-8d3e-9360c7a33687 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.325776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ba3cd-7e8b-4522-8d3e-9360c7a33687 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.326073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ba3cd-7e8b-4522-8d3e-9360c7a33687 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.337485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ba3cd-7e8b-4522-8d3e-9360c7a33687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-689ba3cd-7e8b-4522-8d3e-9360c7a33687 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:42.343081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:42.344026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:42.344855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 323/621] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:42.346033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1da3c-d013-487e-9577-fc711c5680e6 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.347911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1da3c-d013-487e-9577-fc711c5680e6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.347911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1da3c-d013-487e-9577-fc711c5680e6 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.348201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 324/622] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:42.349881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1da3c-d013-487e-9577-fc711c5680e6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.351917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43100b-fa08-4288-aee8-d95a12736225 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.388699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.389318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.389604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.389879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.390569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:42.390876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:42.392341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f7eb6-919f-43a5-b979-b1ca5b3fdae5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:42.406346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 299/623] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:42.411797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.412093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.412520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:42.413609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.434583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:42.436210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:42.461596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1da3c-d013-487e-9577-fc711c5680e6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.500518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.501123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.501453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.501721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.502523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:42.502828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:42.504331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:42.511292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 300/624] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:42.512398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.514082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 325/625] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:42.519004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.567263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:42.645518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f45bc-c0dc-4f17-9236-f49505552e66 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.650093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b2cb9-64f2-4380-afb1-e255629c5d98 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.679524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:42.688393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.689769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.690642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.691146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/94cf8e7d52a3441196ac4b4a63fb13ec/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.694782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.696469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.696818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.697165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.711210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:42.711873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=94cf8e7d52a3441196ac4b4a63fb13ec, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:42.712835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:42.713755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:42.729954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:42.731018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:42.732931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:42.732931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:42.732931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:42.732931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 326/626] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:42.733185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.733984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 301/627] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:42 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/94cf8e7d52a3441196ac4b4a63fb13ec/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:42.739128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:42.883678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27bd3a0-0281-4e7f-90d8-e6d714e38038 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.915503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:42.936534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bbb6e9-a1b1-4d50-93fd-b98a4c65398b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:42.958290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:42.959052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:42.959418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:42.959689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:42.972586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:42.979394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:42.980351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:42.981250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:42.984669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 327/628] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:42.989028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.002165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.003071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.003559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.004034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.004911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:43.005395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.007151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.021570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 302/629] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:43.027657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.035688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.150162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357970d-bd59-4fb9-8ac1-810b4ee01254 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.175286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37408ed-a39e-4822-9193-e59be60983c7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.190421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.191431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.191862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.192398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.193369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:43.193677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.195068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b8948-4964-4744-a28e-8f431f73a08c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.207130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 328/630] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:43.209082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeef203-4380-4609-9d62-65daeb7dd953 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.209663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeef203-4380-4609-9d62-65daeb7dd953 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.209969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeef203-4380-4609-9d62-65daeb7dd953 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.211747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 329/631] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:43.217637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeef203-4380-4609-9d62-65daeb7dd953 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.222050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.223236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.223236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.223705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.233907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.235787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8332c831-7d77-46b5-9028-426cb1212e48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:43.252137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:43.253386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:43.254611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:43.260802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 303/632] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:43.260802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.260802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.260802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.282412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:43.284264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:43.288314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.331765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.344118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeef203-4380-4609-9d62-65daeb7dd953 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.381044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.381806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.382141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.382420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.383850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 304/633] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:43.385199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.401586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.408356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:43.430438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:43.431519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:43.432400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:43.432400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:43.432400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:43.432400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 330/634] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:43.438516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.541556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e42f97-fbec-441e-a4d9-ab5dbdcada6e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.593182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.593897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.594177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.594454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.595210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:43.595522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.596964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.600575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddab0978-26f7-4f68-ab07-369c15344be5 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.607397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.621765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 305/635] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:43.622979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.644757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.645407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.645741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.646013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.658305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:43.663619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:43.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:43.665372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:43.669892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 331/636] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:43.670934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.676439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.759700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd0b9c-da3b-4500-b279-af3960cae174 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.787971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f269e5-d8a1-4db0-88d7-cc8a9c5e7b96 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.801469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.802095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.802418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.802647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.803333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:43.803641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.805064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.819568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 306/637] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:43.824058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:43.829851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.830464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.830744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.831017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.893122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.912366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.913823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.916758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:43.943409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f3c5a-8eb0-4954-8628-81337fb08901 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:43.981080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:43.981801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.982113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.982417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.983240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:43.983550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:43.984972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd489c2-c089-4670-a4a1-ee108574d59f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:43.994833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 307/638] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:43.996173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711632b2-ba15-40c8-93b9-c95cbd280124 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:43.996465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711632b2-ba15-40c8-93b9-c95cbd280124 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:43.996744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711632b2-ba15-40c8-93b9-c95cbd280124 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:43.998608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 308/639] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:44.000017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711632b2-ba15-40c8-93b9-c95cbd280124 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.033827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 332/640] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:44.033827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.124667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711632b2-ba15-40c8-93b9-c95cbd280124 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.144550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0e18e-b4b2-4870-b8e7-dd3072261b5f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.145846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.148762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.164069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.164726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.165008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.165282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.174512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.176010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.186776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.187649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.188168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.188589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-33c333ad-1d57-453e-b7c1-d4f6fbf88ee0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:44.188803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:44.189893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 14:18:44.191040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:44.194680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 309/641] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:44.195976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.196262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.196539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.203707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.205412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.215675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:44.216921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:44.223021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:44.231568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:44.233064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:44.233064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:44.233064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:44.233064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 333/642] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:44.233064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.299996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 310/643] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:44.301164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.342031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8d4e8-9354-4367-8593-7115d29bef66 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.379860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.380839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.381119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.381397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.392527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.393957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:44.407832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:44.408770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:44.409875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:44.409875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:44.409875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:44.409875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 334/644] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:44.409875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.419315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913943e-cb91-4b9f-998b-648a22f76df9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.423856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.432697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.459802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.460414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.460697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.460973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.462019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:44.462331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.463790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.481043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 311/645] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:44.481844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.534020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae9eed-808e-4ff5-93d1-84829df3858b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.571402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.572526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.573400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.574037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.574953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:44.575454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.577430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.591049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f7a7-12fe-42b6-a7bc-657a98e66735 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.594633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 335/646] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:44.595667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.626364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.626989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.627295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.627567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.638064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:44.642700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:44.643726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:44.644642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 312/647] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:44.645390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.677629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.678421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.716917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7d345-ebe5-4b42-800b-656c02d86f57 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.753595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.754203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.754479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.754750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.755421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:44.755728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.756016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b8cb9-aa50-4160-8beb-eed30a195911 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.757098 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.772849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 336/648] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:44.772849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.800685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.801289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.801565 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.801870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.877001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.878395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.886521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6590f6-2d4a-4746-b77a-ebfefd450873 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:44.906989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.916458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:44.924843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:44.925478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:44.925761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:44.926037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:44.934155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:44.935635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:44.946808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:44.953052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 337/649] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:44.954035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:44.995050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 313/650] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:45.000323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.061910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56014e1e-186b-4b78-ac10-417d25e5f681 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.099292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.099907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.100187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.100466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.101124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:45.101428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.102799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.110840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 338/651] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:45.111625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.120942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ea3ce5-ce89-4e22-887b-ee6a0a2ca4eb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.151842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.160626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.161255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.161526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.161795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.165281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.168999 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.170453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:45.181175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:45.182115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 314/652] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:45.182997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.220799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89550c52-598f-4b5f-b341-2abc5060997a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.256189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.256796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.257076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.257355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.258497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:45.258799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.260628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b2a2a-b864-42f5-a673-a26e8394cd4f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.278207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 339/653] 10.222.0.50 () {56 vars in 1137 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:45.279440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7fb69-9bb1-4733-8b1b-5eb7853862b1 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.279724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7fb69-9bb1-4733-8b1b-5eb7853862b1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.279998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7fb69-9bb1-4733-8b1b-5eb7853862b1 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.281181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 340/654] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:45.281870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7fb69-9bb1-4733-8b1b-5eb7853862b1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.287228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e774cd-5427-442a-b88c-2a9fe6ce5129 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.321664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.322292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.322577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.322849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.333416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.334861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:45.349207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:45.350294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:45.351225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:45.351225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:45.351225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:45.351225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 315/655] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:45.351465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.384419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7fb69-9bb1-4733-8b1b-5eb7853862b1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.393212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.394264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.420346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.420892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.421173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.421436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.431548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.432995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:45.447761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:45.448696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:45.449555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:45.449555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:45.449555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:45.449555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 341/656] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:45.449555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.454639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f2ec0-8a23-4cbe-ab5d-159e708faa1b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.490694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.491329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.491604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.491872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.492551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:45.492858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.494288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f616b-435a-4e33-bb08-724ccabcb39c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.506586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 316/657] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:45.507766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.508048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.508320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.526340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:45.527581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:45.555842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd5569-fc9b-4f7d-9af2-f2aca7691b50 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.590457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.591060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.591395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.591670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.593058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 317/658] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:45.593883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.620924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.631822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.661690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.663197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.701939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179be5a-1246-4f35-9ed5-4876367164b5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.736158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.736768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.737049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.737322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.738383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:45.738696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.740110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.751740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 318/659] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:45.752525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.762561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 342/660] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:45.763461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.825689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.856430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:45.869100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47251-b17d-4685-be47-7be8fce23e50 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.870046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27949166-3b5c-45b3-8d2f-2f43e0d8bc12 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:45.905926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.906136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:45.906532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.906711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:45.906819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.906992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:45.907130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.907295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:45.907811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=a587bc4f9082494ab0ab2b6ccbd517ee) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:45.907932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:45.908118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.908237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:45.909542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.909650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:45.915309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 343/661] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/role_assignments?user.id=a587bc4f9082494ab0ab2b6ccbd517ee&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:45.916092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:45.926912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 319/662] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:45.926912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.034373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d51dd7-3a81-4f2b-85d7-03f618907c06 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.039410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac8fd0-2f75-439a-9ca1-61ed3fdb6712 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.059854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.071867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.072539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.072888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.073162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.085804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:46.088083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.088441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.089100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.089100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:46.090551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:46.092061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 344/663] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:46.095952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.097430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.098770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:46.112243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:46.113405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 320/664] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:46.114427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.129004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.219897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c1124-21f2-4dc7-8dc0-29db8dab4cac None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.226954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac999c-0256-4f60-b9db-21972c988a30 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.257010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.257641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.257914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.258181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.264482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.265110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.265386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.265659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.270512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.272016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.273455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.274903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:46.286155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:46.287318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 14:18:46.288408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:46.289055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:46.290087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:46.291152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:46.291152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:46.291152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:46.291536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 321/665] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:46.291536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.296144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 345/666] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:46.296144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.332825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.384349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.404433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6a538-759a-4336-8b2b-352eb2449999 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.418042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8540e148-dd2e-4172-84c9-1561e6b36706 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.441504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.442177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.442462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.442722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.443502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:46.443810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.445258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.456417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.457021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.457298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.457570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.458064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 346/667] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:46.458279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:46.458616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.460000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.460134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.469630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 322/668] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:46.470432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.565568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.571746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a96d0a-3a56-4e09-acd9-f42748b6d4d1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.582424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d4818-3eee-4cc7-be74-c4efa3505eb2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.608837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.609450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.609722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.609995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.610713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:46.611011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.612455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.614698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.620605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.621255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.621538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.622578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.622578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:46.622841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.624830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.635474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 347/669] 10.222.0.50 () {56 vars in 1139 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:46.635474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.641765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 323/670] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:46.646209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.744882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87384d0d-7c0e-45c5-9a57-1500cf2bbc0e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.749186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9db3-16ce-4dab-a71d-f970588562bf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.781642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.782241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.782521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.782790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.783535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=9355f8f80a434ab8af3145f7e82c277c) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:46.783842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.785273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.788049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.788945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.789428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.789772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:46.791860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 348/671] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/role_assignments?user.id=9355f8f80a434ab8af3145f7e82c277c&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:46.792976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.824911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:46.826757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.827601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:46.846859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:46.848112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:46.849374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:46.849374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:46.849374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:46.853095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 324/672] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:46.854067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:46.928169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:46.945816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb03291-dca0-4188-81bf-f6ac208b31b6 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:46.987123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:46.987757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:46.988028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:46.989247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.006666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945710b-a9e1-491c-8a8f-1bf65776710f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.053208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.053847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.054112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.054385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.064686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.066166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.079684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.081193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:47.081613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:47.082445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:47.083357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:47.084661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 325/673] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:47.085342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.112679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:47.215368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373d0e-6082-4734-a5ac-846110f98f03 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.216906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 349/674] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:47.217837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.223551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:47.258031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.258639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.258916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.259215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.259854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:47.260153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.261679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.274645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 326/675] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:47.275579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.352101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9326db-2858-4456-a527-0433cec3a477 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.373038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:47.392965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.393488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb733d-3f45-4447-985c-911f4fb1fee0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.393679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.393918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.394192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.394986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:47.395322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.396877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.410357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 350/676] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:47.411403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.442032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.442780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.443244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.443523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.458511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.460127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:47.476671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:47.477625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:47.478501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:47.478501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:47.478501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:47.482615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 327/677] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:47.483602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.511522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:47.528116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501256f3-d5c0-4d3a-bff1-507b941f8c8c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.564588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.565638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.566126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.566626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.575455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.577881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:47.589950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:47.590915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:47.591907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:47.594419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 351/678] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:47.595519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.610392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a42e3b-8bc0-44a5-af0c-397774c9b929 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.643840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:47.660224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.661282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.661571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.661844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.662579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=d22e2d936518407fba2d0be379598ce6) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:47.662889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.665007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.674550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 328/679] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/role_assignments?user.id=d22e2d936518407fba2d0be379598ce6&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:47.675402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.787263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:47.794141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963b338-cd9b-4e11-8fb1-ba72dcf81736 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.814008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf46faa-2266-4ba2-ba75-210ad05e98b9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:47.862207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.862945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.863262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.863539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.864575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:47.864920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.867389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.894040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 329/680] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:47.904423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:47.927166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:47.927791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:47.928069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:47.928341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:47.928993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:47.929294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:47.932386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:47.956023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 352/681] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:47.961419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.066556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9fb6b-4b48-4acf-bc34-8e033ff72a0a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.088755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:48.093921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0c9b-0515-41ea-a13f-c4f558ea045e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.115027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92551b-41a7-4772-9342-552cd92c3c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.115589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92551b-41a7-4772-9342-552cd92c3c79 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.116263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92551b-41a7-4772-9342-552cd92c3c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.116551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92551b-41a7-4772-9342-552cd92c3c79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.128745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:48.137400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.138496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.138997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.139316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.147010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:48.148715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c84832c4-ea0c-4ea4-b59d-4aeb747dd4c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:48.161532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:48.162511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:48.163449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 353/682] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:48.169174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa657745-37f1-450c-befd-a785b0ebdc66 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.169978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa657745-37f1-450c-befd-a785b0ebdc66 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.170253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa657745-37f1-450c-befd-a785b0ebdc66 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.171959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 354/683] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:48.172775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa657745-37f1-450c-befd-a785b0ebdc66 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.215309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92551b-41a7-4772-9342-552cd92c3c79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:48.217331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92551b-41a7-4772-9342-552cd92c3c79 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:48.301471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa657745-37f1-450c-befd-a785b0ebdc66 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.344231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.344856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.345135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.345409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.346116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:48.346421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:48.347838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 330/684] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:48.349372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.349905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.350374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.351502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:48.367994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 355/685] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:48.368919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.371452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:48.373180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:48.390980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:48.411130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:48.460716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 331/686] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:48.462227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.489218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a9d30-39d7-459e-84e6-e68849190f98 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.543378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.544089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.545021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.545607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.558316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:48.560148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:48.585312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:48.586291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:48.587168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:48.587168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:48.587168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:48.589201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 356/687] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:48.589552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.612700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0524a-43b2-42a9-8b14-de8c466477ca None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.656511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.657072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.657347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.657617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.658517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=511ecf1b54294d088d4737c1841cad5b) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:48.659083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:48.660642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:48.671368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 332/688] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/role_assignments?user.id=511ecf1b54294d088d4737c1841cad5b&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:48.672466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.681050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:48.723959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:48.743714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a7793-c3d3-4ca0-b9c2-dd91041a1b37 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.817900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc835a-d8b5-4a4e-8d55-6fc062f3f480 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:48.819924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.820597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.820880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.821153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.841000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:48.847050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:48.849284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:48.851279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:48.856400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 357/689] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:48.858160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:48.859036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:48.859552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:48.860020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:48.860937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:48.861466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:48.863108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:48.872183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.879539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 333/690] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:48.892324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:48.995136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb243f19-7ffe-4611-a80a-541e23bc891c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.029335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.036244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.036882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.037161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.037432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.038123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:49.038437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.039911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.050218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 358/691] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:49.060800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.132083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.142688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6996a51-6a2e-44f1-a7ff-4ef33fdad8d5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.180713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.181936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.182324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.182613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.188829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e654-1f1d-4f3e-8e9a-029da49b87c1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.191270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.192981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-81632224-d97e-48ea-bd2b-7f24310f78f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:49.206731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:49.207755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:49.208640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 334/692] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:49.214106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb57d8-de44-42f7-8fe4-4cb2070fc01f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.214506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb57d8-de44-42f7-8fe4-4cb2070fc01f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.214792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb57d8-de44-42f7-8fe4-4cb2070fc01f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.216756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 335/693] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:49.217508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb57d8-de44-42f7-8fe4-4cb2070fc01f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.228071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.228710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.229052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.229325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.242521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.244127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-9314167b-4704-4529-a5b0-f962cabe0065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:49.261180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:49.264036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 359/694] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.266135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.285373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:49.286773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:49.331907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.333697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb57d8-de44-42f7-8fe4-4cb2070fc01f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.371430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 360/695] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:49.371430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.382636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.383513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.383992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.384411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.385302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:49.385771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.387382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.410625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 336/696] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:49.416858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.441150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.477137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11dd51-1ad5-47a5-8c72-1afee60f5569 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.514524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.515198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.515351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.515629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.516277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:49.516583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.518190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.531511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 361/697] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:49.532572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.538843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696e751-33a4-43b2-996a-c4e196a30198 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.569374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.581987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.582975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.583473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.584052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.585085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=94cf8e7d52a3441196ac4b4a63fb13ec) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:49.585545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.587448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.601564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 337/698] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/role_assignments?user.id=94cf8e7d52a3441196ac4b4a63fb13ec&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:49.602786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.643878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2a457-6fdd-4274-84c8-2b7f97b07556 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.687984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.688656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.688940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.689215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.716259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.718061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:49.757634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:49.758960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:49.760021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:49.760021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:49.760021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:49.761971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 362/699] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:49.783569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.784763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.812200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a17d-a6a4-41a6-ae9b-d605c8e351c1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.854969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.856001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.856402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.857324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.866173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.868016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:49.881429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:49.882455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:49.883697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 338/700] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:49.888733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:49.924728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:49.943728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2354bc-dc51-4cda-a8ee-d305a73d7827 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:49.981078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:49.982585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:49.982745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:49.983156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:49.983966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:49.984240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:49.985769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:49.999524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 363/701] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:50.002481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3d53c-c84f-41b3-9f3f-2cf3866e08e3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.005112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.042650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.043219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.043509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.043829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.045131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:50.045443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.048767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.052085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:50.073209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 339/702] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:18:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:50.078694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.133485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65acc9-3cf0-4760-b434-e57ff9fefab9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.179008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b4a6-68f7-42a2-bc07-55df269fce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.180063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b4a6-68f7-42a2-bc07-55df269fce3a None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.180541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b4a6-68f7-42a2-bc07-55df269fce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.181602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b4a6-68f7-42a2-bc07-55df269fce3a None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.182452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b4a6-68f7-42a2-bc07-55df269fce3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.184846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97b4a6-68f7-42a2-bc07-55df269fce3a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.215860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04a417-dae2-47cf-b582-0c63e32b0225 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.218276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 364/703] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:18:50 2022] POST /identity/v3/users => generated 254 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:18:50.220891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32a36-cffb-4faa-963c-7084e04144d7 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.221412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32a36-cffb-4faa-963c-7084e04144d7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.221868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d32a36-cffb-4faa-963c-7084e04144d7 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.223472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 365/704] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:50.230332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.230631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.230874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.239828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:50.262224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:50.263556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:50.295342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.296370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.296853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.297321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.375522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:50.390994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.394165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.434211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 366/705] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:50.457665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.539537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 340/706] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:50.545937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.574402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb8d4c-8715-4155-844d-1406b93b7af5 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.613232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:50.619970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.621054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.621541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.621930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.622616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.624790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:50.639059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:50.640134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:50.640955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 367/707] 10.222.0.50 () {56 vars in 1129 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:50.647218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.678570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:50.688486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630986d-8922-4310-ba4f-2822cea6011d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.734192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.734831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.735155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.735437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.746397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.747941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:50.765628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:50.767929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:50.770050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:50.770050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:50.770050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:50.770050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 341/708] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:50.779116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.779393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264e0d8-5448-450b-96e7-a6cb9b932a2b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.823455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.824524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.824959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.825390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.826093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:50.826397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.828562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.835437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 368/709] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:50.840205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.913196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066aef4d-53cd-43e2-b976-b628bffe14cd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:50.921378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:50.959734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:50.960960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:50.961462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:50.961950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:50.962975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:50.963497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:50.965506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:50.981505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 342/710] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:50.988193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:50.996517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d502ff5-a350-424d-a176-c30bc10aa230 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:51.004333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:51.061523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:51.062182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.062460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.062724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.063458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:51.064498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:51.065873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609c400-7ce9-4d33-a12f-62e6757c4dfa None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:51.077627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 369/711] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:51.155438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c549a7-8aca-483e-bc4a-0cd2858617d1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:51.174860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d815b3-06ec-4fec-8276-06eaa49112b7 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.175207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d815b3-06ec-4fec-8276-06eaa49112b7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.175492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d815b3-06ec-4fec-8276-06eaa49112b7 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.177196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 370/712] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:51.182666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.182956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.183248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.220611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:51.221825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:51.230471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:51.231192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.231450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.232267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.245891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:51.250039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:51.270131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:51.271180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:51.272061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:51.279341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 343/713] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:51.288172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:51.364350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 371/714] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:51.376930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:51.390070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:51.423898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3ede1-8bd5-41d8-9c0c-88efb7d3581d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:51.464802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:51.465571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.465854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.466665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.467431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:51.467736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:51.469206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:51.485141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 344/715] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:51.493126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:51.509964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:51.590792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e29b-5fcb-41f0-b1b8-aa95b60a69b8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:51.659632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:51.660209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.660502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.660790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] PATH_INFO: `/v3/services/network` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.661407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:51.662786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:51.677757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:51.680158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:18:51.682205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:51.688918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459611f9-d323-41b3-8796-9f78e9452ae1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:51.689086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 372/716] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:51 2022] GET /identity/v3/services/network => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:51.689086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:51.728860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:51.749275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:51.749948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.750227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.750494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.751246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=a587bc4f9082494ab0ab2b6ccbd517ee) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:51.751550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:51.754320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:51.770154 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 345/717] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:51 2022] GET /identity/v3/role_assignments?user.id=a587bc4f9082494ab0ab2b6ccbd517ee&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:51.847155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e23d9-448b-42ea-909e-f6fffeedf454 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:51.887511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:51.906561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:51.907332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:51.907645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:51.907915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:51.908583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] RBAC: Adding query filter params (name=network) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:51.908884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:51.910415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d31de4-ce50-445f-b737-72577675c267 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:51.915775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 373/718] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:18:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:51.921784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:52.075219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01723c5-1988-4ffc-99fc-4bcb1d3d7cc4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:52.113026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:52.113638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:52.113914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:52.114182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:52.114872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] RBAC: Adding query filter params (type=network) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:52.115201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:52.116562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f52f85-49dc-475f-b46d-af84ad646565 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:52.122697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 346/719] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:18:51 2022] GET /identity/v3/services?type=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:52.639254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176a3f3-5b7d-4088-857c-440fe6de78aa None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:52.639800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176a3f3-5b7d-4088-857c-440fe6de78aa None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:52.639800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176a3f3-5b7d-4088-857c-440fe6de78aa None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:52.641535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 374/720] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:52.667920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0027a8-74b5-4795-86a8-530a99a45c4d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:52.668572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0027a8-74b5-4795-86a8-530a99a45c4d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:52.668572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0027a8-74b5-4795-86a8-530a99a45c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:52.695003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7f0027a8-74b5-4795-86a8-530a99a45c4d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:52.696266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7f0027a8-74b5-4795-86a8-530a99a45c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:52.768039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:52.792026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 347/721] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:52.800588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcd73f-f6cb-4547-9645-b49fabf5708f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:52.800883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcd73f-f6cb-4547-9645-b49fabf5708f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:52.801132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcd73f-f6cb-4547-9645-b49fabf5708f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:52.802716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 375/722] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:52.808530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:52.808859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:52.809070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:52.834861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:52.836105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:52.865243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcd73f-f6cb-4547-9645-b49fabf5708f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:52.964545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 348/723] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:52.987548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.083550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdcd73f-f6cb-4547-9645-b49fabf5708f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.154422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.155461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.156107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.156576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.174802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c717b7-2116-47bf-8de4-39a3e0b97a80 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.183875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:53.192202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:53.193166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:53.194005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:53.196642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 376/724] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:53.205229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.228705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:53.270559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.272143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.272789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.275210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.295451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:53.295833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:53.303181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:53.304225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:53.305095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 349/725] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:53.312959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.425991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99eefc-e401-42d8-89ea-bee8e93aa2a4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.428154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbda441-1efe-4c10-bbf2-4f58b7b19a5d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.465326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.466738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.467234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.467643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.468446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:53.468912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:53.470416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869c4c-31d7-499f-a76c-a0bdfe2189ab None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:53.475958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.476782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.477693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.478086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.479267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:53.479641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:53.481233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd38a7-37a4-4b4c-8ad6-9f5fc06a0369 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:53.485941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 350/726] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:53.487503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79c342-517f-4db4-b2ef-48d68234b0ee None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.487934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79c342-517f-4db4-b2ef-48d68234b0ee None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.488352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79c342-517f-4db4-b2ef-48d68234b0ee None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.490226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 351/727] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:53.491241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79c342-517f-4db4-b2ef-48d68234b0ee None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.491906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 377/728] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:53.495008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.495510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.495917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.517362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:53.518856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:53.537474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:53.599083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 378/729] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 14:18:53.600371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.618189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79c342-517f-4db4-b2ef-48d68234b0ee None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.637235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:53.657038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.657787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.658301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.658781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.670503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:53.673006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:53.689052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:53.690029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:53.690913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:53.690913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:53.690913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:53.692981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 352/730] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:53.694368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.717542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283d5bc-6ecb-48d7-befa-c17cbcc819b6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.756392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.757298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.757759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.758188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.770390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:53.772309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:53.792565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:53.793569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:53.794459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:53.794459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:53.794459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:53.801474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 379/731] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:53.802846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.810777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:53.826920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf180ca-ff86-42e1-adf6-2d1d19234e56 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:53.875902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:53.875902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:53.875902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:53.875902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:53.876319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:53.876762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:53.903879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 353/732] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:18:53 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:53.906519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:53.926167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:53.956145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f797-27ec-46c0-97eb-18799ab61f63 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.049242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.052467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.053037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.055179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.056197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:54.056729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.059493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.085488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 380/733] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:54.088645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:54.105875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c20c9-a3c7-47f4-8684-e0491f4b3d2f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.143826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.144433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.144710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.144982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.145622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:54.145923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.147533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.166192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 354/734] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:54.167519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:54.179778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:54.209193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102acf-227d-434c-8b38-0bffe13aadd6 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.253359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.254195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.254542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.254884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.263217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.264677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:54.277139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:54.278127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:54.279611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 381/735] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:54.288573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:54.291694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:54.333384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f5838-e45a-480e-95bc-e885d0370d70 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.396249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.396882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.397172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.397427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.407333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.408762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d8a58497-8de2-4e65-ab74-f7ebdd8d4e99 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:54.421612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:54.422623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:18:54.423796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:54.427536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 355/736] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:54.428941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f635-751f-41ae-b8ac-e5360640f246 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.429280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f635-751f-41ae-b8ac-e5360640f246 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.429879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87f635-751f-41ae-b8ac-e5360640f246 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.432353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 356/737] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:54.433125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f635-751f-41ae-b8ac-e5360640f246 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:54.456984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c5ee-a1cd-4c8b-b5c0-ac2dcaf0b403 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.505435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.506929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:54.507846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.507846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.508126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.508906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:54.509139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.510624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2afb4b-1b31-4a40-9205-5c253d404fe9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.540815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 382/738] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:54.542466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.542749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.543021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.552329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87f635-751f-41ae-b8ac-e5360640f246 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.565765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:54.566965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:54.591398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.592426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.592995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.593629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.594556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:54.595138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.597018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.622306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 357/739] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:54.623384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:54.632848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:54.645827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 383/740] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:54.646701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:54.750792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290ab5d-34dd-46b5-96c2-27eb5053abaf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.766108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554dee0-f098-48d9-beb6-540d057a49fd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:54.784699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:54.796398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115e433-be94-445b-a644-e159390c4106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.797049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115e433-be94-445b-a644-e159390c4106 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.797360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115e433-be94-445b-a644-e159390c4106 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.797605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115e433-be94-445b-a644-e159390c4106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.809319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:54.809966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.810239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.810693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.881367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115e433-be94-445b-a644-e159390c4106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.882824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115e433-be94-445b-a644-e159390c4106 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.883628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:54.895556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:54.897018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:54.994876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 358/741] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:54.996437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54cd97-b1d6-45de-a8f5-8b96d46b58f9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:54.996722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54cd97-b1d6-45de-a8f5-8b96d46b58f9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:54.997499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54cd97-b1d6-45de-a8f5-8b96d46b58f9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:54.998903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 359/742] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:55.003446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54cd97-b1d6-45de-a8f5-8b96d46b58f9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.014198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 384/743] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 14:18:55.014198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.037389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.112921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.121118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54cd97-b1d6-45de-a8f5-8b96d46b58f9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.125369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a902b-fc6e-470c-91cb-3a7fe6de5239 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.159585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda34f2c-ca9b-4c98-bd1e-a6b4a33301a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.160237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda34f2c-ca9b-4c98-bd1e-a6b4a33301a2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.160518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda34f2c-ca9b-4c98-bd1e-a6b4a33301a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.160793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.160910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda34f2c-ca9b-4c98-bd1e-a6b4a33301a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.161375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.161635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.161902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.172540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda34f2c-ca9b-4c98-bd1e-a6b4a33301a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:55.172807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.174312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-fda34f2c-ca9b-4c98-bd1e-a6b4a33301a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:55.177439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:55.178401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 360/744] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:55.179963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.180738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.180738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:55.194735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:55.195761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:55.196637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:55.196637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:55.196637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:55.196637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 385/745] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:55.196637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.204145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:55.205354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:55.287285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 361/746] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 14:18:55.287285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.308077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.326880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6d95-b2f7-458d-bf81-31c98620f6bf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.365056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.365692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.365999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.366610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.378570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.380143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.382163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:55.397092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:55.398120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:55.399808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:55.399808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:55.399808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:55.399808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 386/747] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:55.399808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.411985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61f08cb-f2f0-467b-b51b-5e709187e917 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.449358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.449981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.450301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.450581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.451657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:55.451971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.453702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.467744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 362/748] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:55.467744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.526624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547fc6b-be76-41c7-a460-160e9010cdbe None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.556484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.566710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.567079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.567385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.567658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.568308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:55.568610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.570000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.582298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 387/749] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:55.583894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.589750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2089b-c7a5-44b4-9459-7ee004ed51de None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.628002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.628616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.628898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.629169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.629920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.631332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.645926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 363/750] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:18:55 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:18:55.646749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.657258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.716265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6b48a-8f17-4fd0-ac25-3ef278e03075 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.767529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.768911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.769420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.769696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.770414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:55.770722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.772563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.790761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 388/751] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:55.792005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.815234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b109-4ada-447c-950c-7ad11f33a2af None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.868107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.869099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.869582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.870066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.881125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.886180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.888118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:55.909018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:55.910022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:55.910888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:55.910888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:55.910888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:55.912718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 364/752] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:55.913699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:55.924935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1444d-6934-4c19-acd5-172a98a519f1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:55.945988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:55.975247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:55.976802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:55.977592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:55.978052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:55.987955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:55.989846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:56.002937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:56.003887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:56.004757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:56.011324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 389/753] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:56.012468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.027262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9edf9-99c6-4d1b-bec8-d1bd8659d203 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.065488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.066055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.066336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.066607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.073860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.075371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:56.087017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:56.088153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:56.089131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 365/754] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:56.089602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.132036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb155d-6de6-423c-8e37-2301145be4a2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.146224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:56.173306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.174174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.174636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.175053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.175966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:56.176416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.178032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.192840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 390/755] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:56.194138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.199982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76847a-baa8-4bae-aef7-319dbbbdc8a1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.230405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:56.247134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.247134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.247134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.247134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.247559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:56.247559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.247559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.258609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 366/756] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:56.259685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.306252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ea06-8204-48fe-bbdf-645b1fca1eb1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.346233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.346857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.347154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.347432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.348191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:56.348496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.350398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420184cc-184d-421e-885d-41f008d36e56 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.371655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 391/757] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:56.371655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c2d12-4aa6-4cb0-93cf-fb58a57ac93c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.371655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c2d12-4aa6-4cb0-93cf-fb58a57ac93c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.371655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c2d12-4aa6-4cb0-93cf-fb58a57ac93c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.371655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 392/758] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:56.371655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c2d12-4aa6-4cb0-93cf-fb58a57ac93c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.381338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f6424-ac26-45d9-bff4-881bd7de85c0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.395842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:56.425541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.426173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.426489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.426759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.435304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.437047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:18:56.450921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:56.451942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:56.452871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 367/759] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:56.453447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.573929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7941b0-ab3e-4c32-9687-ae42080bc79a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.601322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:56.611464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.612109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c2d12-4aa6-4cb0-93cf-fb58a57ac93c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.612232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.612416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.612684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.613530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=94cf8e7d52a3441196ac4b4a63fb13ec) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:56.613815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.615377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb898d2-a59a-4423-aff4-351f2062c759 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.629347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 368/760] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/role_assignments?user.id=94cf8e7d52a3441196ac4b4a63fb13ec&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:56.629347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.629347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.629347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.655261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:56.656613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:56.667169 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.667786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.668062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.668329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.669285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=b644a4d4db88473989cd032bbc16b678) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:56.669589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.671176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.678042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 393/761] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/role_assignments?user.id=b644a4d4db88473989cd032bbc16b678&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:56.679108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.709211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:56.799100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 369/762] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:18:56.876492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e5d24-ebc8-4f7c-b1bc-2fc861d40573 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:56.881007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.922489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:56.923209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:56.923509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:56.923785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:56.924657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:56.924929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:56.926527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:56.955156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 394/763] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:56.958272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:56.987460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:57.083857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723058c0-6d1d-48cc-83e9-271dd19fe2d6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.158565 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.159134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.159416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.159685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.160278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.165831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.166430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010dbff-f5d8-41b6-a323-cfcd73c28ab3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:57.191376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:57.192402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:18:57.193247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:57.193970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 370/764] 10.222.0.50 () {56 vars in 1129 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:57.200200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:57.224452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.226043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.226915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.229207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.257252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.259247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.276480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:57.286528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:57.287714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:57.288805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:57.288805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:57.288805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:57.295887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 395/765] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:57.304274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:57.398327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fa612-7c81-46cc-8881-f1ba80edaf0b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.438866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.439584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.439957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.440331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.441116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:57.441524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.443043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.449215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 371/766] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:57.456190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:57.458145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:57.502120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a29d4-dcf0-487b-8aee-5ee77f5c14e2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.539005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.539665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.539946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.540258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.543111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:57.543111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.543111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.555879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 396/767] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:57.562143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:57.665503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:57.679209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab45dd-bfe8-445d-919a-c092e3b0872f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.731942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f2a9c-a0d7-4112-bb64-9d81989a4faa None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.737210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.737818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.738097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.738369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.739003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:57.739335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.740712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.752393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 372/768] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:57.759120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:57.787161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:57.808381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.809026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.809300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.809613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.820432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.821886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:18:57.834393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:57.835347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:57.836914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 397/769] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:57.845465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:57.910879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facfa16a-f8f6-4238-bef7-77d258af6717 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:57.959709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:57.960428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.960713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.960986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.961751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] RBAC: Adding query filter params (interface=public, service_id=d649330adebe446c9cb11f516488fde2, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:57.962012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:57.963624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d8329-0d45-4743-838c-a4d5a4b73dde None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:57.979274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 373/770] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/endpoints?service_id=d649330adebe446c9cb11f516488fde2&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:57.980775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbad8-cc17-43c0-877f-1868925f2820 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:57.981069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbad8-cc17-43c0-877f-1868925f2820 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:57.981396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbad8-cc17-43c0-877f-1868925f2820 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:57.983134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 374/771] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:57.983813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbad8-cc17-43c0-877f-1868925f2820 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:58.009241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5a5eb-82f5-4bb1-82dd-93955addb2e8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.014057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:58.046140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:58.046849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.047184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.047526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.048221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:58.048525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:58.049993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0164-b6a6-46e9-b8b1-03f0bc10f5a4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:58.068902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 398/772] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:58.070219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.070571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.070850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.089633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:58.091119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:58.101168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbad8-cc17-43c0-877f-1868925f2820 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.133713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:58.139955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:58.140779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.141207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.141620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.142470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:58.144128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:58.155379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 375/773] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:18:57 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:58.156628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:58.163616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 399/774] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:18:58.189911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:58.358928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:58.381329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f2d94-6faa-4bbe-997e-bec826e33c92 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.390684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2239-b969-425a-8d42-86cbfebdee4f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.425799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:58.426874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.427415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.427906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/a587bc4f9082494ab0ab2b6ccbd517ee/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.430783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=a587bc4f9082494ab0ab2b6ccbd517ee, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:58.431669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:58.432373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.432754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.433345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:58.433461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] PATH_INFO: `/v3/services/network` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.434132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:58.436702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:58.446695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:58.448009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:18:58.448867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:58.449600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 400/775] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:18:58 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:58.454726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:58.458210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 376/776] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:18:58 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/a587bc4f9082494ab0ab2b6ccbd517ee/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:18:58.510305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:58.582526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb4972-1937-45b2-9a6c-c30f90766983 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.628969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:58.630014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.630492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.630963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.631809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] RBAC: Adding query filter params (name=network) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:58.632294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:58.633996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:58.640174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 401/777] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:18:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:58.645547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:58.768657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5745a9aa-5b81-47e8-ab1a-58a28843d198 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.807800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:58.808993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.809497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.809985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.810798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] RBAC: Adding query filter params (type=network) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:58.811303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:58.814447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfb444-6f0c-4e02-a664-769ab534e88f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:58.820481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 377/778] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:18:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:58.828218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:58.893103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469e81a-9eea-458d-833c-b6d6cec78da4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.893651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469e81a-9eea-458d-833c-b6d6cec78da4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.894107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469e81a-9eea-458d-833c-b6d6cec78da4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.896266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 378/779] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:18:58.902160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39025d8d-65f3-4c7f-9e33-b679c67f0ed2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:58.902775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39025d8d-65f3-4c7f-9e33-b679c67f0ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:58.903782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39025d8d-65f3-4c7f-9e33-b679c67f0ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:58.935620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-39025d8d-65f3-4c7f-9e33-b679c67f0ed2 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:58.939115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-39025d8d-65f3-4c7f-9e33-b679c67f0ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:58.942356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:58.995000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc297682-8600-496e-bac5-f8e333a21e3b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:58.997010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:59.038632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.039308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 379/780] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:18:59.040204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.040734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.041081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b6285-09b1-471f-a302-266f2e83eeba None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.041378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b6285-09b1-471f-a302-266f2e83eeba None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.041622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.041857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b6285-09b1-471f-a302-266f2e83eeba None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.042761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] RBAC: Adding query filter params (interface=public, service_id=37bf0b4cdc9c4f439660221916dbeb89, region_id=RegionOne) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:59.043360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:59.043534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 380/781] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:18:59.046041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:59.052853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.053455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.053933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.059147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 402/782] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:18:58 2022] GET /identity/v3/endpoints?service_id=37bf0b4cdc9c4f439660221916dbeb89&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:59.060067 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.082684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:18:59.083920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:18:59.175217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 381/783] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:18:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:18:59.176048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.181887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8113f84-ef55-4d9c-a872-b7d587cc1c01 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:59.238202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.238846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.239178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.239454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.240342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:59.242353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:59.249388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 403/784] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:59.250334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.270804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:59.297658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:59.337840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3d8f-ed85-4f52-90ac-f2792c94773d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:59.417009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.417688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.417979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.418265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.432796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:59.441222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:59.442168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:18:59.443006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:59.447299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 382/785] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:59.450012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.458261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344fa09-30d5-4d8c-a56c-b18b7a8b9e9b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:59.503610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.504413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.504789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.505181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.519988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:18:59.526717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:59.527745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:18:59.529125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:18:59.530151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 404/786] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:18:59.535980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.595212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de345a7-68ae-4abc-94f1-881bbd5d70ef None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:59.632076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:59.641408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:59.643975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.644660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.645110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.645417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.646439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:59.647388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:59.648946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:59.649370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd382a3-579a-4660-be7a-bef2c2e572fd None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:59.663208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 383/787] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:18:59.667220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.739975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.740523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.740804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.741074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.741750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:18:59.742052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:59.743471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:59.764644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 405/788] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:18:59.780168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.790525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a0421-777b-4b14-b44b-6f28ebe7545d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:18:59.830755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:18:59.831464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:18:59.831934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:18:59.832217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:18:59.845383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:18:59.846953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:18:59.864148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:18:59.865128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:18:59.865983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:18:59.865983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:18:59.865983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:18:59.869484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 384/789] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:18:59.872214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:18:59.930234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:18:59.984790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d0c3d-3b67-41f4-955b-309bbe112527 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.028756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac91980-296e-446d-865c-53fba045c0e8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.053136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.054574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.055031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.055475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.077522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.078136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.078422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.078697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.079372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:00.079820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.081768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.091937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.093687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.094995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 385/790] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:00.104420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:00.126056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:00.127335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:00.128234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:00.128234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:00.128234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:00.136824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 406/791] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:00.143692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.156253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:00.251040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd7139e-3186-4265-ae21-0a4feda58e27 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.253138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:00.321550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.322200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.322480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.322756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.325177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc276405-3d23-4675-ae2a-d763347080ff None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.344997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.346804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:00.367731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:00.369003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:19:00.370178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:00.376774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 386/792] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:00.377691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.402743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.403893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.404189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.404446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.405150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:00.405451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.406827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.420993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 407/793] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:00.428357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.506579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606618f3-72b7-44c5-a0f3-081fd5c6f214 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.543189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.544351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.544632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.544901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.545812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:00.546123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.548800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.578167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 387/794] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:00.588277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.612927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:00.634080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:00.686780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888eee48-a6d7-41b5-bf76-034dfb29e899 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.727020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.727735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.728016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.728283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.740451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.744544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:00.769868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:00.770872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:00.771856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:00.772377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 408/795] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:00.781175 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.811446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ad7788-cdc8-4593-8c99-9031c0fa3860 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.865983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.866549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.866834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.867145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.883987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.885773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:00.901898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:00.902888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:00.904356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:00.904356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:00.904356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:00.906417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 388/796] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:00.911712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:00.915442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f633-1ea2-4a69-ba5f-59abf6574edf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:00.940905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:00.954187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:00.954800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.955080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.955378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.956214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:00.956526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:00.958159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebec988-7bf0-4316-8c26-d0bc02405e36 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:00.982801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 409/797] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:00.984463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984da80-fc4e-4003-9832-30ada69c570f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:00.984667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984da80-fc4e-4003-9832-30ada69c570f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:00.984947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984da80-fc4e-4003-9832-30ada69c570f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:00.986447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 410/798] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:00.987354 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984da80-fc4e-4003-9832-30ada69c570f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.000119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.031863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260a14d-d3c4-4d93-8a03-b0d7e4604d80 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.068095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.068708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.069297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.069583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.070239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:01.070542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.072206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a0b755-0a50-43e3-a0c0-83b9c6c6bf9d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.084546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 389/799] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:01.089356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.089356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.089356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.104421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:01.105619 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:01.109349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984da80-fc4e-4003-9832-30ada69c570f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.146497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.147045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.147352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.147626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.158298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.159772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-da388f09-07b3-4a57-9166-3404b21b5b78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:01.174832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:01.175841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:01.176714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:01.176714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:01.176714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:01.176714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 411/800] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:01.177536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b36b5-86ee-47ed-8b93-dbae87c035be None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.177536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b36b5-86ee-47ed-8b93-dbae87c035be None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.177675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b36b5-86ee-47ed-8b93-dbae87c035be None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.183560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 412/801] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:01.183560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b36b5-86ee-47ed-8b93-dbae87c035be None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.185911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 390/802] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:01.186589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.233724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.253338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.298859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c91bc-7143-4c31-9fab-38cf1d55aeb1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.317619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b36b5-86ee-47ed-8b93-dbae87c035be None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.344430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.345201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.345787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.346077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.346728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:01.347035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.348434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e98fb-5eec-4270-af90-f6c7e3cb4ab2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.353565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.354223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.354506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.354778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.365274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 391/803] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:01.365274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.365274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.365274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.367195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.368701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:01.380918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:01.381995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 413/804] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:01.382876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.384178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:01.385375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:01.459032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 392/805] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:01.460237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.490146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.499682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccaace-6c34-48fe-9101-edd5653919f9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.525904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.537207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.537844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.538127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.538401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.538988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.540780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:01.551858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:01.552841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:01.553668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 414/806] 10.222.0.50 () {56 vars in 1129 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:01.554429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.581436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ab673-fd02-408a-a71b-0cb661af02a0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.622615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.623279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.623566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.623838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.631274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.632724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:01.644101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:01.645095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 393/807] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:01.645972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.666083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a0ecd-ca79-422d-85ce-2850fe680f57 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.703699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.704303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.704582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.704851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.705530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:01.705837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.707305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.720089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 415/808] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:01.725391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.745158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.757293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f27e0a-cd54-4240-8028-613d7d1f9407 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.771676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.794357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.794982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.795285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.795561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.806486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:01.811895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:01.813609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 394/809] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:01.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.834837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a262302-7b92-43b2-95c2-a49988995ef3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.869347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.869946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.870225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.870492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.871147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:01.871446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.872839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.879662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 416/810] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:01.880534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.921056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6121c2e7-ed15-482b-9ff2-7b79b5be41ed None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:01.954036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:01.954626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.954905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.955203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.955882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:01.956185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:01.957607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d3bc7-a563-44d7-a6c0-34dfa4f36c17 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:01.961823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:01.970478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 395/811] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:01.971607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fe45d-2612-44e4-bb6e-94296be4df7e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:01.971895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fe45d-2612-44e4-bb6e-94296be4df7e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:01.972167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fe45d-2612-44e4-bb6e-94296be4df7e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:01.973625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 396/812] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:01.974316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fe45d-2612-44e4-bb6e-94296be4df7e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:01.984429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0fc14-2716-4492-836e-a633f5fd4440 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.006957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:02.020180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.020180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.020180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.020180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/94cf8e7d52a3441196ac4b4a63fb13ec/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.023468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=94cf8e7d52a3441196ac4b4a63fb13ec, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.023468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.037013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 417/813] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:19:01 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/94cf8e7d52a3441196ac4b4a63fb13ec/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:19:02.040572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.088169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fe45d-2612-44e4-bb6e-94296be4df7e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.131937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.132567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.132848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.133122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.133868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:02.134367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.136462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.152266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 397/814] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:02.152266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.170173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff66225-7445-4d94-8549-1edb4cc4a87e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.215785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.216424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.216704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.216977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.217616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:02.217918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.219347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2275ea-126b-467b-b2f6-7ac4b5faae1f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.225009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 418/815] 10.222.0.50 () {56 vars in 1124 bytes} [Mon Aug 8 14:19:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:02.226135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.226422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.226698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.240676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:02.247905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:02.250570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:02.267135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:02.274311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6b704-b7f9-469c-8618-ccebe9d5700d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.310869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.311557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.311834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.312150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/b644a4d4db88473989cd032bbc16b678/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.326605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=b644a4d4db88473989cd032bbc16b678, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.331194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 419/816] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:02.331367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.331458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.355320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 398/817] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:19:02 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/b644a4d4db88473989cd032bbc16b678/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:19:02.356696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.460881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8168f-3389-4d35-8a4f-d70c4123f811 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.491599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d64f38-c591-4e5b-b847-72b6a9ee525b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.508863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.508863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.509171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.509519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.514758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:02.526232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.531073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.542846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.543719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.544119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.544495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.545268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.546871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:02.548963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:02.550016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:02.550891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:02.550891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:02.550891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:02.552978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 420/818] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:02.553948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.574237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:02.584297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 399/819] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:02.585295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.769973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2c6ed-2aa5-4876-9907-72f7bf8ea696 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.817091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2b4c4-6dbd-45e2-a1d7-7211990ad71f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:02.835787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.836412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.836883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.837165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.840803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:02.841140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.842557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.857500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 400/820] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:02.868102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.890590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:02.891372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:02.891657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:02.891936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] PATH_INFO: `/v3/services/network` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:02.892590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:02.894164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:02.903883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:02.904905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:02.905711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 421/821] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:02 2022] GET /identity/v3/services/network => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:02.911328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:02.947358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:03.006923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:03.015687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45367a-58e8-45f0-bc51-c486bb1261fb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.060946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d9d76-174c-41e0-8ead-4a5f69d96599 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.079836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.080508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.080790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.081057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.090253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:03.091759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:03.097450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.098162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.098448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.098720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.099430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] RBAC: Adding query filter params (name=network) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:03.099743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:03.101360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:03.107430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 422/822] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:03.112547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:03.115490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:03.117620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:03.119855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:03.127114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 401/823] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:19:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:03.127114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:03.224521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d264b8d-e4a3-43c5-8d69-1a8eaf5972ee None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.251134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:03.261328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aedadc-ab62-4def-b722-0b37df03e46c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.298006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.298633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.299209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.299893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.300867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:03.301186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:03.303126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:03.305360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:03.311282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 423/824] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:03.316020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.316714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.316996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.317269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.318630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:03.319383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:03.319692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:03.321127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:03.364086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 402/825] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:19:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:03.365898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:03.473008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725d323-5007-42f5-8a70-996d0f35e9f1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.537553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97583a1-ccc0-46e9-a3b5-b3b2af4c8bea None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.544348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.549691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.550195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.550663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.551551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:03.553153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:03.592493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 424/826] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:03 2022] POST /identity/v3/endpoints => generated 337 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:03.612563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.613667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.614194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.614684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.668426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:03.711261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:03.712774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:03.826210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 403/827] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 14:19:03.831778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:03.881545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:03.950643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ba28c-cfc3-429a-8339-45e947d73e0d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:03.986524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:03.987186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:03.987464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:03.987732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:03.999529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:04.000990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:04.016050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:04.016998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:04.017882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:04.017882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:04.017882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:04.022012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 425/828] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:04.023461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.101167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:04.150516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecf875-481f-4ab6-86f0-cc0b0fab4649 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:04.194484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:04.195549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.196028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.196515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.197354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:04.197997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:04.199736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e242b-a999-4973-9223-31423ad8ed61 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:04.217823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 404/829] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:04.223028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.317004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:04.329713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f1f26-c63f-4dbb-b478-fb7c8c2acafb None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:04.364914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:04.365542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.365819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.366087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.369050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba11f9-009b-4df7-91f8-eb206c8bce39 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.369603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba11f9-009b-4df7-91f8-eb206c8bce39 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.370084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba11f9-009b-4df7-91f8-eb206c8bce39 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.372120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 405/830] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:04.375142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:04.375142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:04.378106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.378640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.379155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:04.386571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:04.388203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:04.389075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 426/831] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:04.393944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.404700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:04.406273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:04.487168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 406/832] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:04.498626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8a8b2-1a6a-4783-8f53-8a81a5443532 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:04.506605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.535147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:04.535839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.536017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.536289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.536987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:04.537315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:04.538881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:04.556568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 427/833] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:04.558095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.569119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:04.621358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cd95d-5ab2-49da-adcc-590966645606 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:04.639544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:04.662127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578df08-3bf4-44f0-b227-af67d42f4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:04.663047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578df08-3bf4-44f0-b227-af67d42f4c45 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.663532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578df08-3bf4-44f0-b227-af67d42f4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.663686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad2a5-23ab-4489-8499-79c400bb5a62 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:04.664169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578df08-3bf4-44f0-b227-af67d42f4c45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.676161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578df08-3bf4-44f0-b227-af67d42f4c45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-4578df08-3bf4-44f0-b227-af67d42f4c45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:04.681439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:04.682359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:04.683469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:04.690669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 407/834] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:04.692264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6939f7-df2b-451a-bd62-0686528e7ae4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.692693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6939f7-df2b-451a-bd62-0686528e7ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.693114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6939f7-df2b-451a-bd62-0686528e7ae4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.695585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 408/835] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:04.696297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6939f7-df2b-451a-bd62-0686528e7ae4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.706741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:04.707400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.707682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.707952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.709099 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=a587bc4f9082494ab0ab2b6ccbd517ee) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:04.709404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:04.711235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da0c3b-373e-46a4-b0aa-d208e56c4f86 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:04.718449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 428/836] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/role_assignments?user.id=a587bc4f9082494ab0ab2b6ccbd517ee&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:04.728659 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.728907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.729616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.760867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:04.764064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:04.829236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6939f7-df2b-451a-bd62-0686528e7ae4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:04.863383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:04.884088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 429/837] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:04.894965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:04.895822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:04.896242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:04.896710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:04.897525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:04.897837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:04.900820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:04.906650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:04.909448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:04.917291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 409/838] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:04.925766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.037004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79be024-ff36-4a03-acde-cb796d9cc126 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.055589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee1e98-bf3a-4b9d-af06-71e9a556e5b0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.080766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dbcff-2f04-434d-8448-d669b932279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.081404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dbcff-2f04-434d-8448-d669b932279a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.081678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dbcff-2f04-434d-8448-d669b932279a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.081957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dbcff-2f04-434d-8448-d669b932279a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.093835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dbcff-2f04-434d-8448-d669b932279a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:05.094700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.095587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.096046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.096497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3b4dbcff-2f04-434d-8448-d669b932279a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:05.098457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:05.099396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:05.100214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 430/839] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:05.101297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab1503-9555-4c30-8799-84874f07d3be None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.101297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab1503-9555-4c30-8799-84874f07d3be None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.101297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab1503-9555-4c30-8799-84874f07d3be None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.103134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 431/840] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:05.105624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab1503-9555-4c30-8799-84874f07d3be None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.116992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.119180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6c9b2fdd-dde7-4476-baf4-d458d8aaefc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:05.138961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:05.139958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 410/841] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.145903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.161905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:05.163240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:05.177838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.197554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.221270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab1503-9555-4c30-8799-84874f07d3be None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.247127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 411/842] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:05.247127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.263978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.264633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.264856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.265167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.265791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:05.266156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.267568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.276727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 432/843] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:05.277555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.366610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c8fde-f41b-4c16-9611-b8d03d5f7c5c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.387789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec14182-2cd1-40a7-a014-afefd0ace96e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.404564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.405280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.405555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.405936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.406578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:05.406878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.408342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.412119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.426327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 412/844] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:05.426327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.428645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.429088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.429396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.429671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.430254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.431668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:05.440925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:05.442243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:05.443268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:05.447347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 433/845] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:05.448262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.460896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.542657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4c72-b748-4321-8e0d-1f3a02cd20b9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.552789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14173c-5df0-4d39-8bca-71c1751b3ee4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.580041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.581037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.581491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.581944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.587999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.588625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.588902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.589172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.594421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.596130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.597446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.598874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:05.610361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:05.611312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 434/846] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:05.612229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:05.612917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:05.613785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:05.614630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:05.614630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:05.614630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:05.614630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 413/847] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:05.619432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.670136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.703069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.725173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067abb1a-43ee-4502-ba2b-b54304bbe73f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.729454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9505677-45c1-4ba3-8892-e4d1263018d5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.767462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.768506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.768655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.768847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.769058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.769226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.769479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.769712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:05.769807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.770004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.770389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:05.770694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.771445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.772318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.777364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 435/848] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:05.779198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.786655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 414/849] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:05.787612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:05.919589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7397a00-cd3d-4b38-aa80-fe7314855170 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.922635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59389-bd99-49f4-a03a-76230f0ff61f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:05.935405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.965604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.966199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:05.966824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.967132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.967416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.967795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:05.968110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:05.968388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.968782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:05.969466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:05.969774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.970807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:05.971496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:05.972190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:05.974681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:05.975957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 415/850] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:05.976748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.005214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 436/851] 10.222.0.50 () {56 vars in 1135 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:06.012830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.150019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18d70c-7d79-4a44-95ca-70a14b1b7a6d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.151971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e10654-a558-4edf-9cfc-4e03f36f2770 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.198586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.200185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.200338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.200890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.201204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.201337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.201519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.201999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.210938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.213086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:06.230610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:06.232180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:06.233126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 416/852] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:19:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:06.238190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.252811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:06.285014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.286644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.308358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:06.354907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa76358-a54c-4bed-84f6-c47d73ea124f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.392561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.393412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.393902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.394371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.395354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:06.395885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.397562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.397986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 437/853] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:06.398981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.419082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 417/854] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:06.427218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.476613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:06.511135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a21efe-0f5a-4ca8-8ede-d29bbff44708 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.552411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235dacb-08fc-4856-a70f-6c95918845db None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.556159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.557050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.557514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.558212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.571286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.572760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.575718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:06.591853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:06.592836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:06.594038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:06.594038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:06.594038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:06.594038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 438/855] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:06.599072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.599732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.600014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.600290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.603823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.681380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.683303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.729504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed242d6-61e2-4020-bffd-e4a6225ef00a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.755427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:06.769377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.770324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.770830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.771438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.772385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:06.772889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.774481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.789478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 418/856] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 14:19:06.789797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 439/857] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:06.794820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.795361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.814960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:06.906639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33f1b-3110-416f-80e2-1393ef2b903d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.907843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef302d5-b708-4ba5-92df-60bc08d09976 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:06.948381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.949019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.949299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.949664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.952552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:06.953182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.953458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.953727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.960031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.961221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:06.961477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.962629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8835ac2d-ce28-4dde-bf37-ec84febcac57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:06.974990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:06.975944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:06.982549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:06.982983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:06.983851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:06.986679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:06.986679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:06.986679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:06.987204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 440/858] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:06.987204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345547c-c8f3-4610-9570-d0ac3933b0cb None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:06.987204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345547c-c8f3-4610-9570-d0ac3933b0cb None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:06.987204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345547c-c8f3-4610-9570-d0ac3933b0cb None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:06.987204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 441/859] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:06.988054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 419/860] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:06.988270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345547c-c8f3-4610-9570-d0ac3933b0cb None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:06.988427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.036299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.068644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.097051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345547c-c8f3-4610-9570-d0ac3933b0cb None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.103718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e75b8b-72e9-4ba1-b5d7-381527ccb67e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.133047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.133656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.133942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.134219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.134847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.135178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.136532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b238c2-f336-49d9-a122-ab86b63633c7 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.139202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.139810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.140088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.140368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.141051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.141361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.142771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.148758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 442/861] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:07.154731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.155025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.155311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.162066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 420/862] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:07.162066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.182469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:07.184506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:07.268674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a42f4-5000-4e40-a56f-a5726f4c662f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.272784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 443/863] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:07.273497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.307504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.317134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.323679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.324421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.324760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.325486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.333784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.335287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:07.348177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:07.349119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 421/864] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:07.350009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.395528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28847ebb-af08-40d1-ae29-f2c386f87204 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.431403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.432896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.433340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.433757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.434635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=94cf8e7d52a3441196ac4b4a63fb13ec) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.435108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.436827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.443740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 444/865] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/role_assignments?user.id=94cf8e7d52a3441196ac4b4a63fb13ec&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:07.444697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.477763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0df98-3d99-4dc0-9ac8-6309d83037e5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.515729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.516456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.516739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.517015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.517612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.519013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:07.529112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:07.530334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:07.531465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 422/866] 10.222.0.50 () {56 vars in 1132 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:07.538325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.579792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.586268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac36f6a-2345-4018-8b5b-209a2fcc29b1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.606293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.624852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.625633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.626054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.626439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.627381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.627793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.629545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.643389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 445/867] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:07.649261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.663723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0981a-711f-4951-81e8-bae9088f27eb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.707595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.708465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.708896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.709343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.710301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.710616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.712076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.718224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 423/868] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:07.723855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:07.768044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eefb4-dee4-42f7-9e63-3e890d40f879 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.815456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.816082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.816365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.816646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.817648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=b644a4d4db88473989cd032bbc16b678) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.817962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.819610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934fe1c3-d33c-4300-aa85-82237ad4adca None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.832622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 446/869] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/role_assignments?user.id=b644a4d4db88473989cd032bbc16b678&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:07.843484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec280b00-bea3-40fe-bffe-e39bc8687a5f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:07.867530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:07.889524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:07.890501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:07.890965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:07.891437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:07.892633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:07.893106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:07.894701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d922b9a-3edc-460b-a3c8-1097b9eaa500 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:07.907267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 424/870] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:08.136079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350d0ef-e931-4281-b38e-a09eed5982d7 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:08.136262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350d0ef-e931-4281-b38e-a09eed5982d7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:08.136545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350d0ef-e931-4281-b38e-a09eed5982d7 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:08.138174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 447/871] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:08.143358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb4481-d855-4e99-a9ab-0517d5c5ae65 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:08.143641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb4481-d855-4e99-a9ab-0517d5c5ae65 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:08.143911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb4481-d855-4e99-a9ab-0517d5c5ae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:08.162794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-38eb4481-d855-4e99-a9ab-0517d5c5ae65 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:08.164015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-38eb4481-d855-4e99-a9ab-0517d5c5ae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:08.238628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 425/872] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:08.260742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350d0ef-e931-4281-b38e-a09eed5982d7 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:08.284824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:08.373461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350d0ef-e931-4281-b38e-a09eed5982d7 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:08.410783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db43a8-d772-4d08-bad1-5e8a420de77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:08.411386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db43a8-d772-4d08-bad1-5e8a420de77f None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:08.411669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db43a8-d772-4d08-bad1-5e8a420de77f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:08.411925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db43a8-d772-4d08-bad1-5e8a420de77f None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:08.412637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db43a8-d772-4d08-bad1-5e8a420de77f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:08.414148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db43a8-d772-4d08-bad1-5e8a420de77f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:08.434501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 448/873] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:08 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:09.594713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edc6d9-1a6a-41c2-9b60-7b82e2b03be9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:09.595630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edc6d9-1a6a-41c2-9b60-7b82e2b03be9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:09.596059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edc6d9-1a6a-41c2-9b60-7b82e2b03be9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:09.597807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 426/874] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:09.603605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:09.604468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:09.604947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:09.624280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:09.625802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:09.703956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 449/875] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:09 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:09.726524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edc6d9-1a6a-41c2-9b60-7b82e2b03be9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:09.833930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edc6d9-1a6a-41c2-9b60-7b82e2b03be9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:09.848959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:09.871021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:09.871722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:09.875624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:09.875624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:09.875624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:09.875624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-cd67851c-6cfd-4438-b9cc-908badc99e27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:09.884614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:09.885785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:09.887323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:09.890175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 427/876] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:09.896386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:09.906357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:10.004849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b87dc-77e8-409b-817d-2bcc066cc39b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.005166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb441cde-9d20-4faa-95f1-735a49f65045 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.005458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb441cde-9d20-4faa-95f1-735a49f65045 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.005732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb441cde-9d20-4faa-95f1-735a49f65045 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.007354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 428/877] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:10.012859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9a90-0e22-45bf-a9d1-d82f5ce6bf13 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.013143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9a90-0e22-45bf-a9d1-d82f5ce6bf13 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.013421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9a90-0e22-45bf-a9d1-d82f5ce6bf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.031886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-581e9a90-0e22-45bf-a9d1-d82f5ce6bf13 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:10.033251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-581e9a90-0e22-45bf-a9d1-d82f5ce6bf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:10.044491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.045106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.045380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.045648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.046286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:10.046589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:10.048016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ce0ef-8062-4661-bac6-cbd893093de6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:10.053080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 450/878] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:10.054286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a1fc-fe0a-4dd9-aca0-cab3850dc2f7 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.054567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a1fc-fe0a-4dd9-aca0-cab3850dc2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.054840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a1fc-fe0a-4dd9-aca0-cab3850dc2f7 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.059182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 451/879] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:10.059319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a1fc-fe0a-4dd9-aca0-cab3850dc2f7 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.119348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 429/880] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:10.120608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.120908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.121686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.139851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:10.141028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:10.142307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:10.177543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a1fc-fe0a-4dd9-aca0-cab3850dc2f7 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.227115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 430/881] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:10 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:10.227115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.228775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.229398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.229680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.230391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.231380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:10.231690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:10.233879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:10.240682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 452/882] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:10.243935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.277836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:10.360430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac275f0d-fe98-4951-8151-bc4648b0fbe1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.375443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ec11b-258c-46c9-b648-bd5b98605435 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.400923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278363ae-8336-497c-831e-ff348611822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.401761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278363ae-8336-497c-831e-ff348611822f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.402141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278363ae-8336-497c-831e-ff348611822f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.402559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278363ae-8336-497c-831e-ff348611822f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.404893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:10.420837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278363ae-8336-497c-831e-ff348611822f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:10.423182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39760d5d-21ed-4161-8d95-a86671768349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.423986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39760d5d-21ed-4161-8d95-a86671768349 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.424407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39760d5d-21ed-4161-8d95-a86671768349 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.424818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39760d5d-21ed-4161-8d95-a86671768349 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.425624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39760d5d-21ed-4161-8d95-a86671768349 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-278363ae-8336-497c-831e-ff348611822f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:10.426355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:10.427270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:10.428089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:10.428717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39760d5d-21ed-4161-8d95-a86671768349 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:10.432494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 453/883] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:10.436101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278363ae-8336-497c-831e-ff348611822f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.455879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 431/884] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:10 2022] POST /identity/v3/services => generated 269 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:10.561867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278363ae-8336-497c-831e-ff348611822f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.568151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c839d2-add5-499f-9297-91222e6b90a8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.568432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c839d2-add5-499f-9297-91222e6b90a8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.568712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c839d2-add5-499f-9297-91222e6b90a8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.570338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 432/885] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:10.578118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.578637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.578915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.602943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:10.604511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:10.609868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.610561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.610911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.611276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.612013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:10.612380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:10.613906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:10.624463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 454/886] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:10.630459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.702355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 433/887] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:10 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:10.707214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:10.724576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.759404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65aa6f-5155-4f37-ac40-73eb69f2af30 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.763435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:10.795586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.796227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.796508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.796779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.815494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:10.817091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:10.832610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:10.833809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:10.834756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:10.834756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:10.834756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:10.838586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 455/888] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:10.839499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.845411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e39f19-a01b-42af-9546-d9bc2cc5017a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.897724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:10.898324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:10.898895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:10.899214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:10.900205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:10.901593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:10.915317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:10.916409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:10.917244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:10.921834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 434/889] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:10.924195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:10.960473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcab1e-826a-4a51-925f-29a6ec369955 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:10.963617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.000568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.001466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.001466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.001663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.002311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:11.002611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.004030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.017496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 456/890] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:11.022754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.052582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.065916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b646ba-d7f5-4c69-ae05-371e5ba26cef None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.112024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.112790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.113581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.113956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.114954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:11.115376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.117088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.128311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 435/891] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:11.129958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.144865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9343206-36b5-4662-b7c7-4053bce63ead None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.183335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.184412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.184978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.185448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.193773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.195617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:11.209565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:11.210533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:11.211520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:11.214147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 457/892] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:11.222733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.242169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069a4be-2a76-4688-9f77-84e9af8d3ed7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.253387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.281035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.281913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.282199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.282477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.283716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:11.284033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.285726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.295060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 436/893] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:11.300819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.323070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.355981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073af2aa-6464-4e80-87e6-f8b096961c13 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.394322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.395329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.395844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.396356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.397839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:11.398373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.400561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.425806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 458/894] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:11.431266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1e8df-5405-4594-b76a-8e26790974c5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.431405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.471227 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.472783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.473221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.473633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.474750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] RBAC: Adding query filter params (interface=public, service_id=69b1354380314681935babab6a095279, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:11.475600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.477449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.485064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 437/895] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/endpoints?service_id=69b1354380314681935babab6a095279&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:11.490249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.540414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.568745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef0e09-2e63-4dfb-8037-56145008256c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.602659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.605974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.606955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.607418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.608149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.625993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9e7c4-bca4-463d-9264-aa6c6716d02e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.673742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.674352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.675176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.675457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.676375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.678463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.687722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 438/896] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/services => generated 1268 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:19:11.696678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.698223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:11.795573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:11.818767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 459/897] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 14:19:11.824820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:11.946058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1be479-3db6-45a0-99d5-f4670cb6c650 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:11.981154 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:11.981836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:11.982126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:11.982376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:11.994277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:11.995761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:12.002944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:12.012033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:12.013955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:12.015073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:12.015073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:12.015073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:12.015073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 439/898] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:12.018954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:12.126081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e460aa-7b28-4726-b420-687609fa9183 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:12.161524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:12.162134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.162415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.162684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.163355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:12.163656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:12.165028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:12.177802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 460/899] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:12.183220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:12.292407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aef5c2-d1d7-4157-8d83-4731faf5ce4a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:12.328497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:12.329140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.329515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.329792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.337017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:12.338478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6574e061-1a83-40f0-a1b8-41aafbaa1f4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:12.350438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:12.352465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:12.354571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 440/900] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:12.358992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:12.404871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:12.441459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52030f-33c5-4e9f-82aa-986010a9a794 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.441794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52030f-33c5-4e9f-82aa-986010a9a794 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.442118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52030f-33c5-4e9f-82aa-986010a9a794 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.443988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 441/901] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:12.451411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.451818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.452199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.471969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:12.473308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:12.473830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6054765-3961-4dd6-8ef6-c500e783f2b4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:12.515070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:12.515983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.516410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.516826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.518102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:12.518559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:12.520591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:12.537685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 461/902] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:12.544796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:12.559139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 442/903] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:12.559139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:12.575815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:12.674221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad5321-23f6-47c7-97f2-27497c2ed4ad None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:12.676140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:12.682049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadae1b-057c-4f68-a819-11d38a37fbc5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:12.713932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:12.714708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.715536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.715802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.716506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] RBAC: Adding query filter params (role.id=8b8106f1821b495f824e139e04670a3b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=b644a4d4db88473989cd032bbc16b678) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:12.716815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:12.718452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0fb1a-389c-46f1-9fa5-ae6d06a5c3b6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:12.718799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:12.719521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.719862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.720207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.720863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:12.722211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:12.724859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 462/904] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:12 2022] GET /identity/v3/role_assignments?user.id=b644a4d4db88473989cd032bbc16b678&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b8106f1821b495f824e139e04670a3b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:12.726319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670c22-f6cf-4594-9d49-ef01a37aa695 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.726599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670c22-f6cf-4594-9d49-ef01a37aa695 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.726875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d670c22-f6cf-4594-9d49-ef01a37aa695 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.728283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 463/905] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:12.733933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:12.734210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:12.734567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:12.752637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 443/906] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:12 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:19:12.764538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:12.766278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:12.862207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 464/907] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:12.884792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:12.944345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:13.001036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d67d4-b58f-4960-93ca-d17c5d35ffdc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:13.036835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:13.037476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.037755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.038030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.038623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:13.040026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:13.048899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:13.049824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:13.050703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 444/908] 10.222.0.50 () {56 vars in 1132 bytes} [Mon Aug 8 14:19:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:13.056172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:13.152179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:13.161171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d430e1-79fe-49e1-90f0-c4fd45a7c4d9 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:13.197841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:13.198375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.198650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.198924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.200167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:13.200468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:13.201846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:13.207325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 465/909] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:13.213100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:13.322367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed255441-548c-44f5-9767-21fd15d807c7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:13.336494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:13.358832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:13.359473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.359752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.360023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.360716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:13.361025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:13.363083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:13.368654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 445/910] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:13 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:13.375102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:13.482828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc883b-2fe7-452c-b550-06760e6e09c4 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:13.518438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:13.519432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.519933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.520401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.521304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] RBAC: Adding query filter params (interface=public, service_id=29b6eebacb774c81a2f77f10337ea875, region_id=RegionOne) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:13.521882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:13.523488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940b537-75a8-4aab-8c61-fcb3c20a1d1a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:13.529791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 466/911] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:19:13 2022] GET /identity/v3/endpoints?service_id=29b6eebacb774c81a2f77f10337ea875&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:13.535783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:13.643867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984566c4-f935-4f2e-af82-3ee1951ddf9d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:13.678477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f0b227-9b6d-4cc2-8c29-9a07a4061764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:13.679083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b227-9b6d-4cc2-8c29-9a07a4061764 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.679384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b227-9b6d-4cc2-8c29-9a07a4061764 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.679655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f0b227-9b6d-4cc2-8c29-9a07a4061764 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.680255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0b227-9b6d-4cc2-8c29-9a07a4061764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:13.681612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f0b227-9b6d-4cc2-8c29-9a07a4061764 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:13.689068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 446/912] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:19:13 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:19:13.850645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7b52-6214-4242-bc88-3ab4aaafbb21 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.851187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7b52-6214-4242-bc88-3ab4aaafbb21 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.851267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e7b52-6214-4242-bc88-3ab4aaafbb21 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.852705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 467/913] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:13.864003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:13.864594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:13.865073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:13.888354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:13.889543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:13.930509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:13.965518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 447/914] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:13.989501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7b52-6214-4242-bc88-3ab4aaafbb21 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:14.012305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:14.102293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e7b52-6214-4242-bc88-3ab4aaafbb21 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:14.138285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:14.138914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.139221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.139494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:14.140076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:14.141491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:14.150509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:14.151502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:14.153433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 468/915] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:14.158450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:14.262441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567a41b-2994-49c9-9d8e-cafc4979d57e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:14.297290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:14.297898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.298177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.298501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:14.299155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:14.299464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:14.300835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:14.305944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 448/916] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:14.312090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:14.406721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:14.423423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9dbb5-8547-417e-97a5-da6652846849 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:14.458946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:14.459596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.459874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.460144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:14.460778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:14.461082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:14.462448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39250bdf-6b59-4244-a022-8b7860bd3f51 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:14.468941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 469/917] 10.222.0.50 () {56 vars in 1114 bytes} [Mon Aug 8 14:19:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:14.474282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:14.523884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:14.582734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d73979-a9b3-4e9e-9434-948c83720c12 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:14.617863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cab26e-7614-46be-a471-d0fdb70cc2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:14.618760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cab26e-7614-46be-a471-d0fdb70cc2b0 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.619219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cab26e-7614-46be-a471-d0fdb70cc2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.619639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cab26e-7614-46be-a471-d0fdb70cc2b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:14.620466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cab26e-7614-46be-a471-d0fdb70cc2b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:14.622505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cab26e-7614-46be-a471-d0fdb70cc2b0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:14.645859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 449/918] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:14.841559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16df02e-5cd7-4ffb-9116-5e184c7b627f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.841940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16df02e-5cd7-4ffb-9116-5e184c7b627f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.842120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16df02e-5cd7-4ffb-9116-5e184c7b627f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:14.843720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 470/919] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:14.849305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6399c-19aa-4794-b531-35ccbaf6cc00 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.849775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6399c-19aa-4794-b531-35ccbaf6cc00 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.849850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6399c-19aa-4794-b531-35ccbaf6cc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:14.877931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-61f6399c-19aa-4794-b531-35ccbaf6cc00 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:14.879619 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-61f6399c-19aa-4794-b531-35ccbaf6cc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:14.957860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 450/920] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:14.980050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16df02e-5cd7-4ffb-9116-5e184c7b627f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:14.997046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8300b-2e07-4f4b-ba97-46679db49233 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:14.997701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8300b-2e07-4f4b-ba97-46679db49233 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:14.998202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8300b-2e07-4f4b-ba97-46679db49233 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.000026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 451/921] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:15.005740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.006039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.006313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.025280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:15.025944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:15.043668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.107990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16df02e-5cd7-4ffb-9116-5e184c7b627f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.113814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 452/922] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:15.137267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.153251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.154232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.154707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.155220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.166660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:15.172323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:15.173236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:15.174054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:15.176648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 471/923] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 14:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:15.182841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.194983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.248166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94a7f0-ed50-4b22-8277-d5b7bf4866bf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.286359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.286967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.287278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.287555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.288188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.289517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.297244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b77ab-a1bb-4b43-b0b3-291195581b21 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:15.300423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:15.301484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:15.302365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:15.305889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 453/924] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:15.306728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.320664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.337154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.338115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.338601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.339063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.340522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:15.341035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.342617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.356797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 472/925] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:15.358635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.427318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0ac5b-c449-4cbc-afce-64b7fda79630 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.434821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.472932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bb167-32f4-4697-83f7-e46cf51a4528 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.474542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.475384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.475764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.476138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.477276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:15.477276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.478835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.490106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 454/926] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:15.490912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.511196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.512130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.512562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.512987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.525569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.527779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:15.543671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:15.544867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:15.545742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:15.545742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:15.545742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:15.552346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 473/927] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:15.553462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.578774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.618288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374df80-b107-4a0a-8707-db2709bac97c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.666521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5dde85-892e-4a27-a407-80280fd600df None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.672246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.673121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.673659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.674147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.675547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:15.676097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.678534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.691395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 455/928] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:15.693195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.695443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.717571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.718556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.719072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.719587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.721030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:15.721808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.724138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.741745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 474/929] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:15.747693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.817053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27648e-5e06-463e-b09a-ab79bc3740f9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.854030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.855598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.856289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.856821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.857986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] RBAC: Adding query filter params (interface=public, service_id=2e3a3aff45c8463faef8a9524ddded31, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:15.858644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.863398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115cbad-b7ad-4ca3-938d-a55c1963dc03 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.870022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 456/930] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/endpoints?service_id=2e3a3aff45c8463faef8a9524ddded31&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:15.871644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38636c6d-aeb0-4fdf-8382-0757df48d83b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:15.872080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e568e-c973-4e01-a4a9-b993e7eaded4 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.875972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e568e-c973-4e01-a4a9-b993e7eaded4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.876479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e568e-c973-4e01-a4a9-b993e7eaded4 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.878127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 457/931] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:15.878930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e568e-c973-4e01-a4a9-b993e7eaded4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:15.882806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:15.913973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:15.914708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.915052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.915431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.923287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:15.924766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-2f516d1c-2326-4f2a-9685-7242efd0c147 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:15.937046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:15.938041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:15.939033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 475/932] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:15.939731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:15.940309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:15.940309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:15.958484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:15.959691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:15.995478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.001012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e568e-c973-4e01-a4a9-b993e7eaded4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.031202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 476/933] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:16.031202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.043860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.044809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.045304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.045758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.046725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.048519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.055479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 458/934] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:19:15 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:19:16.060484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.141582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.172499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf91488-4bbb-45fa-97a1-0dcb71ed4dc1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.183827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd90188-586d-4cd1-a729-d253d4c920c0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.223823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.224574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.224920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.225285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.225924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.227404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.228312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.229090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.229838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.230361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.231720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:16.232492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.234017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:16.238466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:16.239449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 14:19:16.240333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:16.244621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 459/935] 10.222.0.50 () {56 vars in 1132 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:16.245473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.257201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 477/936] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:16.263195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.300016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.371771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a892d-4b82-4b92-865a-88133817730c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.394880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0da6e-e25e-4953-9245-e194cd76fcb7 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.402000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.410001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.410814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.411214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.411528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.412320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:16.412718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.414235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.420545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 460/937] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:16.426194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.441314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.442483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.442773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.443071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.457250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.458979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:16.478927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:16.479958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:16.481342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:16.481342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:16.481342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:16.483213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 478/938] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:16.488298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.542282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6fa6c-cd23-419a-acaf-e249e40c0a0f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.581250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.582137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.582573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.582994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.583946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:16.584445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.586156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.592234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 461/939] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:16.592978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.597817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.614975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99264985-2cac-4d53-8d01-adf645a055a9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.652743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.653274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.653562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.653823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.654480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:16.654765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.656195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.673758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 479/940] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:16.674785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.675514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.719246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4c71c-9297-40cc-bd28-c0904edb4dd5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.761033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c98ef-146c-484d-bee7-1296aba44cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.762706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c98ef-146c-484d-bee7-1296aba44cc4 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.763242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c98ef-146c-484d-bee7-1296aba44cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.763727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c98ef-146c-484d-bee7-1296aba44cc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.764614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c98ef-146c-484d-bee7-1296aba44cc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.766437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c98ef-146c-484d-bee7-1296aba44cc4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.781364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d460c21-8d7d-42c9-86aa-04083e1adb62 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:16.795736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 462/941] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:16.799761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85f13-604b-4f0b-907c-1b6338f6e655 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.800427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85f13-604b-4f0b-907c-1b6338f6e655 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.800960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85f13-604b-4f0b-907c-1b6338f6e655 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.802949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 463/942] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:16.813219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.813830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.814290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.833615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:16.834315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:16.834595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:16.834915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:16.836723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:16.838206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:16.843971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:16.845615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:16.848423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:16.864113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:16.865112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:16.872358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 480/943] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:16.873674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.943569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 464/944] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:16.963398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:16.999142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5429f-f17d-49ca-b2f0-1482cc49c214 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:17.022838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:17.050186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:17.052305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:17.052743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:17.053166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:17.054030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:17.054520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:17.056281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:17.072522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 481/945] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:17.081065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:17.086687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1d0ca-7de7-4179-8cdf-1117d85fa9b3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:17.126384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:17.127120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:17.127669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:17.128013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:17.128728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:17.131112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:17.133982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING keystone.server.flask.common [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 14:19:17.151323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:17.154005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.sql.core [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 14:19:17.154005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 14:19:17.154005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [parameters: {'id': 'd7be32865fae46a7a79068203c89b4cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 14:19:17.154005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80133) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7be32865fae46a7a79068203c89b4cc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7be32865fae46a7a79068203c89b4cc. Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application conn.query(q) Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 14:19:17.173555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result.read() Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 14:19:17.174457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application next(self.gen) Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application yield resource Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application next(self.gen) Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise self.value Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application session.commit() Aug 08 14:19:17.175618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.session.flush() Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application compat.raise_( Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 14:19:17.176808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 14:19:17.177932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application conn.query(q) Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application result.read() Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 14:19:17.178899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [parameters: {'id': 'd7be32865fae46a7a79068203c89b4cc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:17.180129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:17.181622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d7be32865fae46a7a79068203c89b4cc. Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 465/946] 10.222.0.50 () {60 vars in 1140 bytes} [Mon Aug 8 14:19:16 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:17.184410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:17.212487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6817f9-acf6-4a4f-9d50-dd1e1155ef27 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:17.254569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:17.255306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:17.255637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:17.255857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/b644a4d4db88473989cd032bbc16b678/roles/8b8106f1821b495f824e139e04670a3b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:17.258229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=b644a4d4db88473989cd032bbc16b678, role_id=8b8106f1821b495f824e139e04670a3b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:17.259726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:17.277613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 482/947] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:19:17 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/b644a4d4db88473989cd032bbc16b678/roles/8b8106f1821b495f824e139e04670a3b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:19:17.321196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:17.327939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2f648-d659-45fe-94b2-5a084a1e0b85 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:17.373674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:17.374712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:17.375320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:17.375801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:17.384236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:17.386334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-38889fe0-ac25-42f3-9443-cd61c70d8c20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:17.399789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:17.400742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:17.401660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:17.404545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 466/948] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:17.414051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:17.528410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddeefe-f80f-447a-9cb6-2808e997d0d8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:17.549223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:17.568607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:17.569221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:17.569501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:17.569776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:17.570448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:17.570755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:17.572232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1259f7-fac1-4b3f-be94-17d6d84789cc None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:17.594904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 483/949] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:17 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:18.213598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865455-70e3-4fe6-94a4-7160baa9fe02 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.214093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865455-70e3-4fe6-94a4-7160baa9fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.214093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6865455-70e3-4fe6-94a4-7160baa9fe02 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.218453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 467/950] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:18.223601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68e424-8890-4307-a612-653740f02f33 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.223996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68e424-8890-4307-a612-653740f02f33 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.224170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68e424-8890-4307-a612-653740f02f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.242704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3a68e424-8890-4307-a612-653740f02f33 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:18.243903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3a68e424-8890-4307-a612-653740f02f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:18.314727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 484/951] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:18.336775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865455-70e3-4fe6-94a4-7160baa9fe02 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:18.457646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6865455-70e3-4fe6-94a4-7160baa9fe02 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:18.493885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:18.494789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.497028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.497541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.498349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:18.499988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:18.509990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:18.511355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:18.512405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:18.514821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 468/952] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:18.520351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68e424-8890-4307-a612-653740f02f33 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:18.624122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68e424-8890-4307-a612-653740f02f33 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:18.633904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:18.668294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:18.669259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.669747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.670250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.671111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:18.671673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:18.673313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:18.678867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 485/953] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:18.684369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:18.711532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:18.796744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2863cb-a26e-4432-8e6c-7419c5a52a3e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:18.835121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:18.835736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.836021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.836297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.836936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:18.837244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:18.838627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372f9c9f-fd59-45b3-bc18-bf4fbffa3e8d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:18.845109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 469/954] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:18.852218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:18.962599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d04748-8467-42e0-a543-6eec3fcf4250 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.962949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d04748-8467-42e0-a543-6eec3fcf4250 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.963202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d04748-8467-42e0-a543-6eec3fcf4250 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.965399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 470/955] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:18.970672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:18.970958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:18.971267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:18.974222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e0561-7bd2-473c-9dfe-e1242d70267f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:18.993077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:18.994875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:19.018386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:19.019008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.019313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.019589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.020251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:19.022399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:19.049319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 486/956] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:19.086668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 471/957] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:19.109426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.144180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:19.233102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93301768-4ee5-477b-9b49-e2aedf99d0ff None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:19.268802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:19.269425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.269699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.269966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.270544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:19.271947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-7ad101ec-845e-485a-b56e-7a87ba7a491d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:19.280902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:19.282074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:19.283609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 487/958] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:19.288076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.303811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:19.399904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a8254-9930-42eb-88ff-033714eeaf8b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:19.434638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80e973-abd4-40fc-b7af-8340a5e0891a None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.434983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80e973-abd4-40fc-b7af-8340a5e0891a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.435204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80e973-abd4-40fc-b7af-8340a5e0891a None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.436783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 488/959] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:19.436967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:19.437513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.437793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.438061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.438702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:19.439141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:19.440604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:19.442335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.442622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.442900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.446446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 472/960] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:19.451990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.465607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:19.467160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:19.544734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 489/961] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:19.555740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:19.567737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.584717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615f986-cc77-4431-813f-b33f956a2801 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:19.594090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:19.626629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:19.627388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.627737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.628081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.628807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:19.629190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:19.630631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:19.637470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 473/962] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:19.695717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ef79f-78d6-479f-9bbf-841c30c52cc0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:19.736865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f50ef-1794-4cbd-ad44-8021c6483422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:19.737524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f50ef-1794-4cbd-ad44-8021c6483422 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.737810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f50ef-1794-4cbd-ad44-8021c6483422 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.738089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f50ef-1794-4cbd-ad44-8021c6483422 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.751488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f50ef-1794-4cbd-ad44-8021c6483422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-182f50ef-1794-4cbd-ad44-8021c6483422 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:19.756395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:19.757395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:19.758222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 490/963] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 14:19:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:19.764841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.817238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711f4a-a89a-464d-bd8e-0bd47890dd6f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.817590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711f4a-a89a-464d-bd8e-0bd47890dd6f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.817695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711f4a-a89a-464d-bd8e-0bd47890dd6f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.819537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 491/964] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:19.825822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.826333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.826703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.849984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:19.851294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:19.894360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33777977-186d-4a71-9277-de3efdc9fbd0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:19.926151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 492/965] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:19.938888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:19.939529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:19.939806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:19.940132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:19.940795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:19.941096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:19.942457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:19.952074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 474/966] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:19:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:19.952501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.958421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:19.962825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:19.971651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:20.085663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f2e35-d4c3-484e-a2a3-b5290b23546b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.092531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225478f-8b20-4f48-9a28-3e5cb9926690 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.125569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.126434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.126854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.127322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.128099 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.129493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.129851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.130135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.130416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.130692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.133306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING keystone.server.flask.common [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 14:19:20.142599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.145252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.148989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.sql.core [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 14:19:20.148989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 14:19:20.148989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [parameters: {'user_id': '320c67e2278b4bb599de1aedd5c92e57', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 14:19:20.148989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80134) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application conn.query(q) Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result.read() Aug 08 14:19:20.158151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 14:19:20.159223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application next(self.gen) Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application yield resource Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application next(self.gen) Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise self.value Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application session.commit() Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 14:19:20.160389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.session.flush() Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application compat.raise_( Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 14:19:20.161277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 14:19:20.162322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application conn.query(q) Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application result.read() Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 14:19:20.163342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [parameters: {'user_id': '320c67e2278b4bb599de1aedd5c92e57', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:20.164359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 14:19:20.165322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 14:19:20.166652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 14:19:20.166652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 14:19:20.166652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 14:19:20.166652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 14:19:20.166652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 14:19:20.166652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:20.167029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:20.169695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:20.170851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:20.170851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:20.170851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:20.170851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 475/967] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:20.170851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:20.171828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 493/968] 10.222.0.50 () {60 vars in 1131 bytes} [Mon Aug 8 14:19:19 2022] POST /identity/v3/users => generated 160 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:20.177959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:20.234874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:20.275495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:20.287362 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eb4d59-36e5-44c2-b22d-d7ea85d34982 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.306596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359193b-2193-4224-b5e5-95cc167743f8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.325061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.325751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.325944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.326217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.326858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:20.327233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.328816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.342404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 476/969] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:20.347278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:20.350893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.351652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.352000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.352351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.360989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.362702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:20.374607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:20.375621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:20.376545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 494/970] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:20.382880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:20.484463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd41aa8-1b21-4fb6-8a14-df6c08b51484 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.504861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:20.525375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f15cbd-cccf-475c-aede-1d6bcdc1625b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.530023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.530658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.530935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.531255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.539688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.541611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:20.556944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:20.558241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 14:19:20.559234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:20.563239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 477/971] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:19:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:20.564927 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:20.576774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.577730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.578839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.579361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.580733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:20.581240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.583671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.586708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:20.641896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 495/972] 10.222.0.50 () {56 vars in 1101 bytes} [Mon Aug 8 14:19:20 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:20.708505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc601da2-6b77-4d0e-8178-fc511161c780 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.747251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.747873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.748153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.748426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.749176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:20.749485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:20.750972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:20.775193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 478/973] 10.222.0.50 () {56 vars in 1149 bytes} [Mon Aug 8 14:19:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:20.781473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:20.897135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85f71d-9db4-495f-a277-83cf51290b4c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:20.933750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451c2b2-bba1-471e-9939-0529d1987b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:20.934370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c2b2-bba1-471e-9939-0529d1987b4e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:20.934649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c2b2-bba1-471e-9939-0529d1987b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:20.934924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451c2b2-bba1-471e-9939-0529d1987b4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:20.996561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.006189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451c2b2-bba1-471e-9939-0529d1987b4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.007718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451c2b2-bba1-471e-9939-0529d1987b4e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.109197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 496/974] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:20 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:21.114917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.150125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.183206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb85ee-f94a-42e2-bd62-9cee34c23145 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.183585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb85ee-f94a-42e2-bd62-9cee34c23145 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.183787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb85ee-f94a-42e2-bd62-9cee34c23145 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.185243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 497/975] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:21.190101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.190390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.190671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.210981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:21.212542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:21.220175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191ef41-0728-4350-80cb-1889a3cda4e0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.257081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.257711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.257987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.258261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.269060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.270498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.283382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 498/976] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:21.285513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:21.286709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:21.287642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:21.287642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:21.287642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:21.291311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 479/977] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:21.292470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.307676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.318888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.396981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f4cd3-7cea-4e6d-ad4d-eaa0623ca104 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.402862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.419901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cccb9-84fb-4305-be49-f24f18e44073 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.434651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.435282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.435619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.436055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.436623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:21.436933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.438409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.456012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 499/978] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:21.457413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.459033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.459714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.460002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.460269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.460854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.462217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:21.471653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:21.472869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:21.473962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 480/979] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:21.478428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.549563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.574290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc7f2c-f89d-4efe-9b8c-d572114259cf None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.584298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92822e7a-b4f0-4226-ba32-93553bda99f7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.612386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.613017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.613323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.613593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.620361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.620529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.621011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.621290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.621563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.622181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.622281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:21.622544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.623858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.628971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 481/980] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:21.634215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:21.638961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:21.640076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:21.641126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 500/981] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:21.645635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.655908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.745190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085f968-172d-422d-98ad-604dad89e01f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.750564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1d01aa-a600-48dd-a14a-f9617862f88c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.779223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.779849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.780131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.780404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.781035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:21.781342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.782709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490cf3a-a097-49a5-bdee-4e5847973394 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.785455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.786073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.786353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.786627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.787363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:21.787660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 482/982] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:21.787755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.788842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d8d63-cc97-430f-9cf3-60b36aaad88f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.789129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d8d63-cc97-430f-9cf3-60b36aaad88f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.789228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e7ab1-6cb1-4dc8-9b47-be8728113a6e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.789403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d8d63-cc97-430f-9cf3-60b36aaad88f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.790867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 483/983] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:21.793410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d8d63-cc97-430f-9cf3-60b36aaad88f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.804678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 501/984] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:21.805921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.806223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.806532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.813116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.826067 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:21.827277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:21.879603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:21.895077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 502/985] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:21.895948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:21.903451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d8d63-cc97-430f-9cf3-60b36aaad88f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:21.938594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:21.939132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:21.939413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:21.939682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:21.940415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] RBAC: Adding query filter params (interface=admin, service_id=2e3a3aff45c8463faef8a9524ddded31, region_id=RegionOne) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:21.940721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:21.942113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:21.947516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 484/986] 10.222.0.50 () {56 vars in 1249 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/endpoints?service_id=2e3a3aff45c8463faef8a9524ddded31&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:21.948325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:22.011252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984184de-d904-46ad-84b9-515ca9a26ce4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:22.041360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:22.046428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:22.047029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:22.047331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:22.047598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:22.048319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] RBAC: Adding query filter params (role.id=8b8106f1821b495f824e139e04670a3b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=b644a4d4db88473989cd032bbc16b678) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:22.048656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:22.050052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:22.051478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f988e-0128-45d0-9169-404dfbf3bf4d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:22.055844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 503/987] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:21 2022] GET /identity/v3/role_assignments?user.id=b644a4d4db88473989cd032bbc16b678&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b8106f1821b495f824e139e04670a3b => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:22.056614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:22.091391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354d000-b44d-4288-bf28-536429d45c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:22.092047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d000-b44d-4288-bf28-536429d45c44 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:22.092312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d000-b44d-4288-bf28-536429d45c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:22.092590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354d000-b44d-4288-bf28-536429d45c44 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:22.093257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d000-b44d-4288-bf28-536429d45c44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:22.094678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354d000-b44d-4288-bf28-536429d45c44 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:22.123124 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 485/988] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:21 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:22.190654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f199-bfe8-46ad-a818-71537fd9be77 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:22.238646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb8ab-62c1-415b-bdc1-133c64c1ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:22.239172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb8ab-62c1-415b-bdc1-133c64c1ddcc None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:22.239450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb8ab-62c1-415b-bdc1-133c64c1ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:22.239722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb8ab-62c1-415b-bdc1-133c64c1ddcc None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:22.240342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb8ab-62c1-415b-bdc1-133c64c1ddcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:22.241737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb8ab-62c1-415b-bdc1-133c64c1ddcc None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:22.250129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 504/989] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:19:22 2022] GET /identity/v3/services => generated 1727 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:19:24.141645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54633d27-550d-42da-9dec-77df085c073f None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.142162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54633d27-550d-42da-9dec-77df085c073f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.142400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54633d27-550d-42da-9dec-77df085c073f None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.143567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f632a93-442d-4f35-aca0-40779f33de2f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.143959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f632a93-442d-4f35-aca0-40779f33de2f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.144059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 486/990] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:24.144241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f632a93-442d-4f35-aca0-40779f33de2f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.149484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 505/991] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:24.149612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5efef4-1a7e-4c44-a437-21d266005488 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.149612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5efef4-1a7e-4c44-a437-21d266005488 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.149612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5efef4-1a7e-4c44-a437-21d266005488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.150729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176156f3-9faf-417b-a3a4-4cb1e6d6f0bd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.151014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176156f3-9faf-417b-a3a4-4cb1e6d6f0bd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.151316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176156f3-9faf-417b-a3a4-4cb1e6d6f0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.156486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.169071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4d5efef4-1a7e-4c44-a437-21d266005488 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:24.170567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-176156f3-9faf-417b-a3a4-4cb1e6d6f0bd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:24.170919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4d5efef4-1a7e-4c44-a437-21d266005488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:24.171788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-176156f3-9faf-417b-a3a4-4cb1e6d6f0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:24.227883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.248206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 487/992] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:24.251125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 506/993] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:24.252390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12b612-1ec5-4322-88a6-b09f736ed43a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.252675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12b612-1ec5-4322-88a6-b09f736ed43a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.252946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12b612-1ec5-4322-88a6-b09f736ed43a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.254310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 488/994] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:24.260006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.260292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.260567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.271207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12b612-1ec5-4322-88a6-b09f736ed43a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.279075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:24.280279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:24.347714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 507/995] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:24.348814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.380094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12b612-1ec5-4322-88a6-b09f736ed43a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.409737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.415263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.415892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.416174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.416445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.417031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.418498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:24.427233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:24.428406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:24.429393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 489/996] 10.222.0.50 () {56 vars in 1105 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:24.430174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.457923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6e180-9a73-4963-b86c-c80e37a2bb36 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.474257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.492552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.493181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.493457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.493728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] PATH_INFO: `/v3/services/compute` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.494305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.496196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:24.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:24.505492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:24.506318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 508/997] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:24.507238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.531766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ef3d1-7327-469b-943e-7741eb323bf5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.566399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.567022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.567332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.567610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.568436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.569556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:24.578527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:24.579718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:24.580534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 490/998] 10.222.0.50 () {56 vars in 1126 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:24.581297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.606132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487b939-c3fd-4060-98d2-e5f5da5b71ce None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.639113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.639680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.639799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.639972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.640249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.640868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] RBAC: Adding query filter params (name=image) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:24.641201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.642775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.647805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 509/999] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:24.648576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.689055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98a8b9-e561-4c3f-b7dd-d9614e7c3199 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.694271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.721988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.722590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.722868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.723163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.723791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:24.724093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.725450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.730589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 491/1000] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:24.731545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.751230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada1efd-685b-4a7a-a538-6bfd496346be None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.783968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.784584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.784861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.785134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.785768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:24.786072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.787471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.792964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 510/1001] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:24.793751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.838567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ca42a-d55b-4cbb-871c-e165f3f481b1 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.862535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.875742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.876640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.876926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.877198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.878226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] RBAC: Adding query filter params (type=image) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:24.878305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.879707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.884261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 492/1002] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:24.885035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:24.907621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074eb8f-f307-40a7-a9e2-0347e973afa3 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:24.918637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:24.951170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:24.951896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:24.952185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:24.952464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:24.953119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:24.953427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:24.954843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:24.960800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 511/1003] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:24.966505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:25.006966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45e75-33db-4db5-baae-3ab8bec0c36e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:25.047319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:25.048355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:25.048355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:25.048624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:25.050139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:25.050139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:25.051519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:25.057470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 493/1004] 10.222.0.50 () {56 vars in 1122 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:25.063966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:25.098107 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63e3cd-daf6-4fe6-ac2a-ffef10dbfe22 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:25.120954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:25.135194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:25.136105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:25.136565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:25.137050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:25.137772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] RBAC: Adding query filter params (interface=public, service_id=6c9ca62f5de54d1a83f2863dcb3dad85, region_id=RegionOne) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:25.138080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:25.139558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:25.149262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 512/1005] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:19:24 2022] GET /identity/v3/endpoints?service_id=6c9ca62f5de54d1a83f2863dcb3dad85&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:25.151230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:25.177746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5624a-21b6-480f-8aa9-db09bf35d48d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:25.190526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:25.216661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68ec9d-96c8-42c2-ba71-01a0467c900d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:25.217355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68ec9d-96c8-42c2-ba71-01a0467c900d None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:25.217640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68ec9d-96c8-42c2-ba71-01a0467c900d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:25.217911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68ec9d-96c8-42c2-ba71-01a0467c900d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:25.218578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68ec9d-96c8-42c2-ba71-01a0467c900d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:25.220216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68ec9d-96c8-42c2-ba71-01a0467c900d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:25.238993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 494/1006] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:25 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:19:25.264301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0292e8-ab0c-48b2-8366-4d29efb275c8 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:25.310833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c5eb4-b8fd-429c-8bd5-a89f73e089bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:25.311555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5eb4-b8fd-429c-8bd5-a89f73e089bd None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:25.311838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5eb4-b8fd-429c-8bd5-a89f73e089bd None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:25.312111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5eb4-b8fd-429c-8bd5-a89f73e089bd None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:25.312739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5eb4-b8fd-429c-8bd5-a89f73e089bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:25.314224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5eb4-b8fd-429c-8bd5-a89f73e089bd None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:25.329487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 513/1007] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:19:25 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:19:26.917972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a333e4-62bf-4cc0-b273-259fbc40301b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:26.918510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a333e4-62bf-4cc0-b273-259fbc40301b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:26.918510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a333e4-62bf-4cc0-b273-259fbc40301b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:26.923135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 495/1008] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:26.926017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203b23d-ac1a-4de5-bda0-d3ae390c0d64 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:26.926454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203b23d-ac1a-4de5-bda0-d3ae390c0d64 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:26.926622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203b23d-ac1a-4de5-bda0-d3ae390c0d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:26.945260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c203b23d-ac1a-4de5-bda0-d3ae390c0d64 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:26.946509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c203b23d-ac1a-4de5-bda0-d3ae390c0d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:27.017140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 514/1009] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:26 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:27.039393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a333e4-62bf-4cc0-b273-259fbc40301b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:27.141065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a333e4-62bf-4cc0-b273-259fbc40301b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:27.153937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:27.176911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519731a-377f-47a4-a650-fd710672dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:27.177517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519731a-377f-47a4-a650-fd710672dd28 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.177795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519731a-377f-47a4-a650-fd710672dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.178062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519731a-377f-47a4-a650-fd710672dd28 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.178711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519731a-377f-47a4-a650-fd710672dd28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:27.180102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519731a-377f-47a4-a650-fd710672dd28 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:27.197597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 496/1010] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:27 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:27.264171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4bf5c-946e-40bb-89a9-8cd756fb5c7c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.264508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4bf5c-946e-40bb-89a9-8cd756fb5c7c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.264721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4bf5c-946e-40bb-89a9-8cd756fb5c7c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.266348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 515/1011] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:27.271980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.272268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.272549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.292930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:27.294131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:27.345236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3807-13ca-4b17-b8af-25cfeffce476 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.345573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3807-13ca-4b17-b8af-25cfeffce476 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.345801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3807-13ca-4b17-b8af-25cfeffce476 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.347164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 516/1012] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:27.352358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.352633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.352856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.372740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:27.373942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:27.378541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 497/1013] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:27 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:27.398471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:27.402189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:27.448196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 517/1014] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:27 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:27.471682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:27.514145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:27.517760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66feb8d-c297-4a23-9ce1-de3c98ea99da None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:27.558392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:27.559024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.559330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.559604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.560700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:27.562138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:27.563886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 498/1015] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:27.569320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:27.585474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d96f1-33e2-458f-953f-95e0dd34983a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:27.624338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:27.625117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.625481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.625818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.626469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:27.628130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:27.638378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:27.639434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:27.640272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 518/1016] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:27.645274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:27.668812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:27.677258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d241f23a-c625-4413-bf44-84d60b457e7c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:27.714852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:27.715582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.715923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.716266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.716978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:27.718526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:27.759117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 499/1017] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:27 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:19:27.759333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a12e7-8417-487b-b03d-ed9dc126f802 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:27.798873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:27.799572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.799849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.800118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.800804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:27.801163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:27.802597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:27.814356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 519/1018] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:27.815788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:27.838194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:27.943890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e9988-cab0-487c-a7df-253c261d5990 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:27.980466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:27.981089 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:27.981415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:27.981695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:27.982334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:27.982642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:27.984060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea2ba0-6b30-487f-ada7-f92dd6df3510 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:27.989528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 500/1019] 10.222.0.50 () {56 vars in 1112 bytes} [Mon Aug 8 14:19:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:27.996085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:28.084828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:28.102820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3d033-31cd-4b30-b744-ba593079fa30 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:28.137267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14e8c-474f-4e94-8e0e-64fba8c5224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:28.137882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14e8c-474f-4e94-8e0e-64fba8c5224a None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:28.138166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14e8c-474f-4e94-8e0e-64fba8c5224a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:28.138444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14e8c-474f-4e94-8e0e-64fba8c5224a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:28.139124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14e8c-474f-4e94-8e0e-64fba8c5224a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:28.140512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14e8c-474f-4e94-8e0e-64fba8c5224a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:28.162904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 520/1020] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:29.223824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f559a86-3b5f-40eb-9b70-8d86e7cba822 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.224324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f559a86-3b5f-40eb-9b70-8d86e7cba822 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.224324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f559a86-3b5f-40eb-9b70-8d86e7cba822 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.225647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 501/1021] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:29.230666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.231108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.231302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.249057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:29.250222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:29.327705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 521/1022] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:29.349885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f559a86-3b5f-40eb-9b70-8d86e7cba822 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:29.439241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:29.460315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f559a86-3b5f-40eb-9b70-8d86e7cba822 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:29.494180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:29.494725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.495001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.495855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] PATH_INFO: `/v3/services/image` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.496440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:29.497803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:29.506535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:29.507750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:29.509033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:29.511803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 502/1023] 10.222.0.50 () {56 vars in 1105 bytes} [Mon Aug 8 14:19:29 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:29.514249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:29.580576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:29.617293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c619f9b-f4d7-4dca-afed-7dc745a92581 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:29.652037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:29.652645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.652925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.653195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.653821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:29.654124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:29.655508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:29.665115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 522/1024] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:29.666662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:29.771513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e34d7-77e7-476f-a8e2-e526abc1a327 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:29.806003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:29.806620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.807194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.807469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.808633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] RBAC: Adding query filter params (type=image) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:29.808937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:29.810305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16e8fd-c026-4430-9876-8ddf4eebb34b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:29.820733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 503/1025] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:29 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:29.821993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:29.834384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3064-48b7-4ad6-b1ed-05b4c88f3315 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.834667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3064-48b7-4ad6-b1ed-05b4c88f3315 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.834946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3064-48b7-4ad6-b1ed-05b4c88f3315 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.836451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 504/1026] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:29.841592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.841913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.842125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.860697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:29.861896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:29.865169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:29.939132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 505/1027] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:29.946242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b60025-3d0e-45b5-a9a8-311b684b9ab2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:29.959338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:29.976921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:29.984221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:29.984833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:29.985111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:29.985391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:29.986095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] RBAC: Adding query filter params (interface=public, service_id=5ef0c688d6734e75a6d019a9b71b8216, region_id=RegionOne) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:29.986404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:29.987810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:29.993255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 523/1028] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:19:29 2022] GET /identity/v3/endpoints?service_id=5ef0c688d6734e75a6d019a9b71b8216&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:29.999435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:30.074810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3633e8f-6e03-41cb-a3a7-046edf24dfee None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:30.109050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae70b4-5f01-478e-9852-2aed12a28ff2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:30.114467 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:30.115112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.115406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.115675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.117217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:30.119008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:30.126399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 506/1029] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:30.127356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:30.142054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:30.146995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db411f00-93fe-4a4b-806a-fc1823e81851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:30.147924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db411f00-93fe-4a4b-806a-fc1823e81851 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.148202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db411f00-93fe-4a4b-806a-fc1823e81851 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.148477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db411f00-93fe-4a4b-806a-fc1823e81851 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.149083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db411f00-93fe-4a4b-806a-fc1823e81851 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:30.150477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db411f00-93fe-4a4b-806a-fc1823e81851 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:30.157450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 524/1030] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:19:29 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:19:30.246609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7529689d-6e90-4c2e-98c7-77ae97749b14 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:30.258961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ec21-857c-4407-a3fc-0352bb98576a None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.259879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ec21-857c-4407-a3fc-0352bb98576a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.259879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7ec21-857c-4407-a3fc-0352bb98576a None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.262019 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 525/1031] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:30.267730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.268095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.268353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.287475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:30.288698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:30.289882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:30.290504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.290855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.291048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.293093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:30.293885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:30.320515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 507/1032] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:19:30 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:19:30.356981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:30.374985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 526/1033] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:30.397904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:30.466458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:30.510865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70464e08-453b-4906-8ac0-4e1a89ff619d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:30.547918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:30.548543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.548821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.549090 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.566473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:30.574656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:30.576447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:30.578365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 508/1034] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 14:19:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:30.583026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:30.699319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fcb06-0272-489d-b7cd-377a19df73ce None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:30.735632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:30.736302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.736587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.736862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.737528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:30.737832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:30.739226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:30.747938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 527/1035] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:19:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:30.753743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:30.864040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6089-2e27-4577-9dc3-691b4959dd20 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:30.890281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:30.902034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:30.902647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:30.902928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:30.903231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:30.914685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:30.916176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:30.930680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:30.931673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:30.932810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:30.932810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:30.932810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:30.933512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 509/1036] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:30.937931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:30.992138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:31.043344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c659f-6ef7-4d8b-a45a-37206cdb33d7 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:31.078194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:31.078803 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:31.079080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:31.079377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:31.080069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:31.080371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:31.081791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:31.093879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 528/1037] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:31.099325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:31.202922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f941a-fbaa-423f-b67f-cdf2900fd835 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:31.236589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:31.237219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:31.237496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:31.237767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:31.245876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:31.247274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:31.259141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:31.260096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:31.261656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:31.262739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 510/1038] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:19:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:31.266615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:31.372336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbcf98-2bbe-47b8-b46f-ce085220ab35 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:31.383894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:31.410731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:31.411374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:31.411658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:31.411933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:31.412663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:31.412970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:31.414397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:31.437387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 529/1039] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:19:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:31.438594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:31.476673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:31.547819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736a1d8-b2cb-4074-99fe-f075a9c8ba21 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:31.582769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:31.583405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:31.583688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:31.583965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:31.651525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:31.653005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:31.687605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:31.753747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 511/1040] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:31 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:31.759560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:31.869882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa810a-25ef-4e1e-9689-50f7d249f304 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:31.906774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:31.907429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:31.907704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:31.907972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:31.918671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:31.920139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:31.934791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:31.936022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:31.936900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:31.936900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:31.936900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:31.936900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 530/1041] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:31.942793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.045831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01032efa-c428-4c65-976b-691e7a506428 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.080944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.081570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.081851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.082125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.082765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:32.083071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.084451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa82aa-eed3-4a04-b2ca-a9cae766d0bd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.086842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.102358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 512/1042] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:32.103538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.127595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.212201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fe46e-6ad6-4455-9b25-74caeb72702f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.215268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e1ef4-41a0-4e1d-9ea9-11a2768d7e01 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.215268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e1ef4-41a0-4e1d-9ea9-11a2768d7e01 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.215494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e1ef4-41a0-4e1d-9ea9-11a2768d7e01 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.216899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 513/1043] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:32.221745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.222272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.222272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.241408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:32.242686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:32.251476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.252140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.252396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.253043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.260632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.262102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:32.273359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:32.274709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:32.275643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 531/1044] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:32.280336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.321641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 514/1045] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:19:32.343830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.370701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.393914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda44c2-74ee-41be-bfac-009ffd9b545e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.429963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.430568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.430843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.431139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.432085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:32.432384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.434360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9552f-66c9-42b9-99b1-df6a6f3e5d22 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.447377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 532/1046] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:32.448830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce1bc-c699-4d7b-a50c-f3bb36b744fd None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.449112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce1bc-c699-4d7b-a50c-f3bb36b744fd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.449488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce1bc-c699-4d7b-a50c-f3bb36b744fd None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.450665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 533/1047] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:32.453122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ce1bc-c699-4d7b-a50c-f3bb36b744fd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.453659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a505d-1916-4346-afe2-bff371e7f179 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.480178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.490034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.490682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.490957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.491257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] PATH_INFO: `/v3/services/image` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.491834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.493219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-318e0870-50a3-40ec-a9be-cfcce2733629 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:32.503111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:32.504295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:32.505346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 515/1048] 10.222.0.50 () {56 vars in 1105 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:32.506290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.506290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.506290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.523265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:32.524459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:32.560971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ce1bc-c699-4d7b-a50c-f3bb36b744fd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.592909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 516/1049] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:32.593779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.605623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.606241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.606523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.606786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.608322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] RBAC: Adding query filter params (role.id=8b8106f1821b495f824e139e04670a3b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=a587bc4f9082494ab0ab2b6ccbd517ee) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:32.608322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.610131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.619820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 534/1050] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/role_assignments?user.id=a587bc4f9082494ab0ab2b6ccbd517ee&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b8106f1821b495f824e139e04670a3b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:32.619820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.628867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.707014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23ec33-d2a9-4a81-8102-596f0cf32827 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.731296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.747739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.747963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb77bc-922f-457e-9704-1a5d80cd4490 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.748361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.748643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.748919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.749568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] RBAC: Adding query filter params (name=image) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:32.749874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.751353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.756823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 517/1051] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:32.762866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.795347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.796424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.796710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.797294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.810893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:32.816301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:32.817251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:32.818537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 535/1052] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:32.823456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.888969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:32.890291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e01d64-a860-429c-87d1-ced7aa9530cf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.929415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.930028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.930307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.930578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.931053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15bcf0b-22c5-49e3-bd78-58b11630e46f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:32.931251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:32.931535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.933194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.943079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 518/1053] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:32.944102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:32.973451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:32.973984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:32.974270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:32.974538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:32.975231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:32.975539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:32.976903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:32.985977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 536/1054] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:32.991546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:33.001023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:33.058552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c42cc-1ae6-4001-9f56-ad9c21c271c5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:33.098827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:33.099486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:33.099765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:33.100043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:33.100669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdae47c-8264-476c-9b0a-c42a9f8d05ad None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:33.100770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:33.102085 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:33.122673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 519/1055] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:19:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:33.138049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:33.157751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:33.158410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:33.158726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:33.159017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] PATH_INFO: `/v3/users/30e0e828bf1445bbb1be20e1735800b3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:33.176197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] RBAC: Authorizing `identity:get_user(user_id=30e0e828bf1445bbb1be20e1735800b3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:33.177789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:33.183161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 537/1056] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:32 2022] GET /identity/v3/users/30e0e828bf1445bbb1be20e1735800b3 => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:33.186476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:33.299908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009cd5a-8574-4ed5-92ab-1c37f08cd27c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:33.302235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:33.338679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:33.339355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:33.339638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:33.339910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:33.409999 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:33.411476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:33.517591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 520/1057] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:33.519386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:33.625324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93668ed-77ff-4997-97fb-aaec134b6e0e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:33.659282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de22559-252d-49bd-bae1-184f31fd8c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:33.659914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22559-252d-49bd-bae1-184f31fd8c1a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:33.660190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22559-252d-49bd-bae1-184f31fd8c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:33.660462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de22559-252d-49bd-bae1-184f31fd8c1a None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:33.661619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22559-252d-49bd-bae1-184f31fd8c1a None admin] RBAC: Authorizing `identity:get_project(project_id=5a644ac1066544598d7b3cef416933fb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:33.663041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de22559-252d-49bd-bae1-184f31fd8c1a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:33.664920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 538/1058] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:33 2022] GET /identity/v3/projects/5a644ac1066544598d7b3cef416933fb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:33.671522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:33.674780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:33.780161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9b50b-2c99-4cd0-981f-29f2307b136b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:33.818326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:33.818931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:33.819241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:33.819522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:33.820250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=5a644ac1066544598d7b3cef416933fb, user.id=30e0e828bf1445bbb1be20e1735800b3) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:33.820561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:33.822006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788258c-4072-4416-bd89-13f5a5b9a1ec None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:33.834811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 521/1059] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:33 2022] GET /identity/v3/role_assignments?user.id=30e0e828bf1445bbb1be20e1735800b3&scope.project.id=5a644ac1066544598d7b3cef416933fb&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:34.621063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a6cbe-e7a4-4292-b7c6-a2c7fb728ef1 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:34.621561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a6cbe-e7a4-4292-b7c6-a2c7fb728ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:34.621670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a6cbe-e7a4-4292-b7c6-a2c7fb728ef1 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:34.623228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 539/1060] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:34.628585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:34.628585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:34.629714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:34.647170 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:34.648361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:34.715464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:34.722360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 522/1061] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:34.745399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a6cbe-e7a4-4292-b7c6-a2c7fb728ef1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:34.806009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:34.866071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a6cbe-e7a4-4292-b7c6-a2c7fb728ef1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:34.904176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:34.904831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:34.905108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:34.905368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:34.918082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:34.922730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:34.923862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:34.924686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:34.927416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 540/1062] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 14:19:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:34.929857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.030905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19741df-5ec5-4734-85e8-cf8e20c262e1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.071871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.072476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.072754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.073587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.074249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:35.074552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.075936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1ceee-68e6-4a8a-bf4d-678383594ddc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.088703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 523/1063] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:19:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:35.090145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.194200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4accf3-3f12-4589-aeb8-4f4bf4802f0b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.200904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51f5d-7448-4b06-af6d-586ab2fcbb26 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.201196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51f5d-7448-4b06-af6d-586ab2fcbb26 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.201476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d51f5d-7448-4b06-af6d-586ab2fcbb26 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.202943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 524/1064] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:19:35.204088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.207869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.208155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.208431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.226878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:35.228229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:35.236501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.237339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.237623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.237895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.249426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.251365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:35.266651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:35.267857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:35.268817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:35.268817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:35.268817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:35.272581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 541/1065] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:35.273577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.287909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.301768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 525/1066] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:35.324156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.382686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb24cd-55e7-46f8-8e0c-82b795b56d8e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.421658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.422268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.422545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.422814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.423490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:35.423794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.425152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.433430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f176c50-20c3-4418-9180-501ab7305ac5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.438175 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 542/1067] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:35.444135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.467050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.470839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.471383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.471745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.472007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.535084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.548793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.550346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.551502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d637f-ca69-4285-9085-08cf186c8f66 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.586830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.587612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.587892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.588212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.596427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.597882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:35.611002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:35.612106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:35.613083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 543/1068] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:35.617802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.662195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 526/1069] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:35.668613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.700952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.722592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a086b8-f296-43c8-acb9-5c09aabf45c7 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.756922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.757532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.757810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.758080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.758806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:35.759129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.760599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.765872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.776978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10bd64-a4ec-408a-af99-2d2a56a9485b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.781782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 544/1070] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:35.789861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.815967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.816593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.816866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.817138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.828036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.829486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-180a8e89-fd62-4637-98c2-90cb5d170e77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:35.844311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:35.845410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:35.846311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:35.846311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:35.846311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:35.846311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 527/1071] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:35.846535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e41dd-b9fd-4def-8a68-9e87c660278b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.846812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e41dd-b9fd-4def-8a68-9e87c660278b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.847078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663e41dd-b9fd-4def-8a68-9e87c660278b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.848335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 528/1072] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:35.852093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e41dd-b9fd-4def-8a68-9e87c660278b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:35.900084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470fde2-d720-4c9d-a14c-fdf1ef43d72b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.933919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.934549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.934826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.935111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.942082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:35.946540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:35.948622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:35.959910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663e41dd-b9fd-4def-8a68-9e87c660278b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f18d2446-584c-4b4b-8d13-cf30cebf14bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:35.962771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:35.964489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:35.965538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:35.965538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:35.965538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:35.965538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 545/1073] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:35.965811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.965897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.966119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.983458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:35.984576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:35.996501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:35.997114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:35.997396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:35.997667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:35.998308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:35.998615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.000019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.012237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 529/1074] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:36.013021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.015584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.056224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 546/1075] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:36.079463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.149324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb20b-aed4-4f52-bb2c-1847c0a00b69 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.193768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.194381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.196384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.196384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.196384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:36.196384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.196711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70b5f95-d319-4a7d-a5e2-d7878db3094d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.198397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.209908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 530/1076] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:36.215845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.223583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.239616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.240307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.240663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.241052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.252896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:36.257611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:36.258746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:36.259794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 547/1077] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:36.265249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.293446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.338030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc30d65-4d27-4c93-84c8-d0d9cde59362 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.371774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd12ca-bd6f-433c-b6ef-77cdb9151b65 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.372201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.372811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.373092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.373363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.380070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.381537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:36.393128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:36.394161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:36.395207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 531/1078] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:36.400092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.411213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.411818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.412092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.412361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.413020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:36.413315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.414672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.423425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 548/1079] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:36.429516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.473602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.500607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb237883-4029-44ca-9843-28f3335e3fb4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.522792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.530325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e38661-5980-4001-abae-5b03ae6064a5 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.534666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.535411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.535691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.535958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.536642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:36.536946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.538382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.551228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 532/1080] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:36.557745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.564908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.565597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.566399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.566399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] PATH_INFO: `/v3/users/30e0e828bf1445bbb1be20e1735800b3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.567305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] RBAC: Authorizing `identity:get_user(user_id=30e0e828bf1445bbb1be20e1735800b3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.568749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.570337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 549/1081] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/users/30e0e828bf1445bbb1be20e1735800b3 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:36.576969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.656981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9643e11f-8cc0-4b3c-a490-a8840d76e83d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.677622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c150010-a744-4493-80c4-a8728d8eb9f5 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.688561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.692972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.693634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.693912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.694189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/a587bc4f9082494ab0ab2b6ccbd517ee/roles/8b8106f1821b495f824e139e04670a3b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.696375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=a587bc4f9082494ab0ab2b6ccbd517ee, role_id=8b8106f1821b495f824e139e04670a3b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.697838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.711054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 533/1082] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:19:36 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/a587bc4f9082494ab0ab2b6ccbd517ee/roles/8b8106f1821b495f824e139e04670a3b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:19:36.714081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cce8d-815b-4c89-91c9-5a47da9a4cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.714726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cce8d-815b-4c89-91c9-5a47da9a4cdb None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.715009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cce8d-815b-4c89-91c9-5a47da9a4cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.715314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cce8d-815b-4c89-91c9-5a47da9a4cdb None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.716501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cce8d-815b-4c89-91c9-5a47da9a4cdb None admin] RBAC: Authorizing `identity:get_project(project_id=5a644ac1066544598d7b3cef416933fb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.717973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cce8d-815b-4c89-91c9-5a47da9a4cdb None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.719751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 550/1083] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:36 2022] GET /identity/v3/projects/5a644ac1066544598d7b3cef416933fb => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:36.726053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:36.774156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:36.837960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83964895-0e40-4fc5-9c6e-3b732b47ccb4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:36.875115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427b612-e884-4940-a6ef-d0bd9aadd901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:36.875790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427b612-e884-4940-a6ef-d0bd9aadd901 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:36.876071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427b612-e884-4940-a6ef-d0bd9aadd901 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:36.876346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427b612-e884-4940-a6ef-d0bd9aadd901 None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb/users/30e0e828bf1445bbb1be20e1735800b3/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:36.878286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427b612-e884-4940-a6ef-d0bd9aadd901 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a644ac1066544598d7b3cef416933fb, user_id=30e0e828bf1445bbb1be20e1735800b3, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:36.879750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427b612-e884-4940-a6ef-d0bd9aadd901 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:36.893477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 534/1084] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:19:36 2022] PUT /identity/v3/projects/5a644ac1066544598d7b3cef416933fb/users/30e0e828bf1445bbb1be20e1735800b3/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:19:38.696819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eda448-ef82-4c33-affe-7684af8f3c50 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:38.697382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eda448-ef82-4c33-affe-7684af8f3c50 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:38.697382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eda448-ef82-4c33-affe-7684af8f3c50 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:38.699022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 551/1085] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:38.704199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669af221-e516-49db-b0d5-c9e68aeabe02 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:38.704622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669af221-e516-49db-b0d5-c9e68aeabe02 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:38.704749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669af221-e516-49db-b0d5-c9e68aeabe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:38.725070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-669af221-e516-49db-b0d5-c9e68aeabe02 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:38.726319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-669af221-e516-49db-b0d5-c9e68aeabe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:38.810702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 535/1086] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:38.836581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eda448-ef82-4c33-affe-7684af8f3c50 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:38.899574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cf16d-b025-4108-b09b-39d31fcc7b4a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:38.899903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cf16d-b025-4108-b09b-39d31fcc7b4a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:38.900133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cf16d-b025-4108-b09b-39d31fcc7b4a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:38.906053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 536/1087] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:38.907323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:38.907611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:38.907889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:38.926115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:38.927355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:38.955549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:38.958241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eda448-ef82-4c33-affe-7684af8f3c50 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:38.996451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:38.997367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:38.997658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:38.997939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.002552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 537/1088] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:39.008815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:39.014023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:39.014971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 14:19:39.015918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:39.019286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 552/1089] 10.222.0.50 () {56 vars in 1120 bytes} [Mon Aug 8 14:19:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:39.021090 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.026888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.070311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.126052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d783fc-2548-402f-8109-ad67bcf27e2c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.137393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d8836-bcf9-4026-b200-bdbca714095b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.161355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.161961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.162238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.162510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.163199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:39.163509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.164863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.171007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.171667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.171950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.172224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.172898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 538/1090] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:39.178422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.183605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:39.188330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:39.189419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:39.190313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 553/1091] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:39.195398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.209468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.284574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c66635-5fe6-4509-9222-f5778a59acd3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.291075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.295349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5774da-62a1-47cb-ae29-f96df50eb43f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.318910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.319572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.319851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.320123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.330488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.331060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.331183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.331383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.331653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.332307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:39.332542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.332632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.333936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.342768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 554/1092] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:39.347229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:39.348399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:39.349267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:39.349267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:39.349267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:39.349267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 539/1093] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:39.349497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.353816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.430328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.452417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a008e7-60eb-4182-b566-732b14132434 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.456376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb0ef3-80e5-472f-b006-a35e27a782bf None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.486785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.487367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.487644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.487916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] PATH_INFO: `/v3/users/30e0e828bf1445bbb1be20e1735800b3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.489031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] RBAC: Authorizing `identity:get_user(user_id=30e0e828bf1445bbb1be20e1735800b3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.489135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.489642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.489919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.490188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.490454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.490819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:39.491148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.492056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 555/1094] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/users/30e0e828bf1445bbb1be20e1735800b3 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:39.492491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.498295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.505329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 540/1095] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:39.510833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.526123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.598297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50dfa6-6623-423e-8c1b-edb6c905f385 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.617810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a0573-53be-41cf-ad83-029b52145a3d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.632656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.633262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.633536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.633800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.656456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.657090 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.657371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.657651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.660352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.665132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.667240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:19:39.678641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:39.679796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:39.680961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 541/1096] 10.222.0.50 () {56 vars in 1093 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:19:39.686381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.706935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.708510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.735750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.786102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bbd6b-d2fa-4a92-83a9-fd6c0224c053 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.809647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 556/1097] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 14:19:39.815653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.822336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.822976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.823282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.823552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.824276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:39.824577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.826004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.844439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 542/1098] 10.222.0.50 () {56 vars in 1133 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:39.849870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.914382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.924710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832dd0ab-b89d-4eaf-9fbd-cc0c1c6eda53 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.951305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:39.959407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.960023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.960299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.960569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:39.961208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aba24f-267b-48bb-a8cc-75af3b59eb9d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:39.961665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] RBAC: Authorizing `identity:get_project(project_id=5a644ac1066544598d7b3cef416933fb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:39.963149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:39.964800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 557/1099] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/projects/5a644ac1066544598d7b3cef416933fb => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:39.970800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:39.996797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:39.997409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:39.997730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:39.998007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.068790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.070177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.072755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b68b04-e7ca-416a-8f9b-1fac4636f77b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:40.106674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:40.107403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:40.107598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:40.107867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.108587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=5a644ac1066544598d7b3cef416933fb, user.id=30e0e828bf1445bbb1be20e1735800b3) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:40.108893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.110311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.116097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 558/1100] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/role_assignments?user.id=30e0e828bf1445bbb1be20e1735800b3&scope.project.id=5a644ac1066544598d7b3cef416933fb&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:40.124380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:40.174296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 543/1101] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 14:19:40.180292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:40.244083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:40.287933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207aed5-4d9f-41af-af2b-21ed82f5bfd6 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:40.324585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:40.325141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:40.325423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:40.325697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.337132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.339140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:19:40.353131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:40.354782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:19:40.356031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:19:40.356031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:19:40.356031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:40.356031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 559/1102] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:40.360866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:40.459538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70442c-a491-472b-9cda-3316e2cb1a25 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:40.492738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:40.493340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:40.493618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:40.493888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.494518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:40.494824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.496224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.507461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 544/1103] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:40.513832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:40.612801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f788-d8c8-4aa8-b53c-48d5c1d37860 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:40.625608 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:40.646684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:40.647342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:40.647622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:40.647894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.655018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.656492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:19:40.667639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:40.668584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:19:40.669855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 560/1104] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:19:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:40.674582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:40.696509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:40.780348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701a37f-3bc4-4093-8484-f756e6b75d83 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:40.814974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:40.815663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:40.815875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:40.816140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.816826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:40.817133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.818561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699e125-6eee-4d0a-9534-ec6b7a25b746 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.831005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 545/1105] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:19:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:40.837284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:40.937146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ef3ca-ef0b-40a3-8f8c-0b2a0c858d84 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:40.969465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:40.970063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:40.970340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:40.970612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:40.971359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] RBAC: Adding query filter params (role.id=8b8106f1821b495f824e139e04670a3b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=a587bc4f9082494ab0ab2b6ccbd517ee) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:40.971666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:40.973077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596941a1-4323-4b8d-a838-a7a7029c5786 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:40.978710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 561/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:40 2022] GET /identity/v3/role_assignments?user.id=a587bc4f9082494ab0ab2b6ccbd517ee&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b8106f1821b495f824e139e04670a3b => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:42.181117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77618-9d8f-4e89-b299-d0e35a61a822 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:42.181899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77618-9d8f-4e89-b299-d0e35a61a822 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:42.181899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c77618-9d8f-4e89-b299-d0e35a61a822 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:42.183213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 546/1107] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:42.188334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:42.188718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:42.188891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:42.207183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:42.208372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:42.236753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:42.275372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 562/1108] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:42.299917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77618-9d8f-4e89-b299-d0e35a61a822 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:42.374007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:42.400011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c77618-9d8f-4e89-b299-d0e35a61a822 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:42.433764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913ab76-bf68-4565-8ab8-1cc0be497b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:42.434309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ab76-bf68-4565-8ab8-1cc0be497b7f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:42.434585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ab76-bf68-4565-8ab8-1cc0be497b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:42.434854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ab76-bf68-4565-8ab8-1cc0be497b7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:42.435968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913ab76-bf68-4565-8ab8-1cc0be497b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:42.437392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913ab76-bf68-4565-8ab8-1cc0be497b7f None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:42.438937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 547/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:42.444984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:42.540473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e50fd6-57b2-4182-857c-603c11d7caa3 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:42.573827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51ea8b-5386-48e3-9934-5808144f8196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:42.574424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51ea8b-5386-48e3-9934-5808144f8196 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:42.574707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51ea8b-5386-48e3-9934-5808144f8196 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:42.574972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51ea8b-5386-48e3-9934-5808144f8196 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:42.575638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51ea8b-5386-48e3-9934-5808144f8196 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:42.577085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51ea8b-5386-48e3-9934-5808144f8196 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:42.598202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 563/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:19:42 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:44.640596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b650b5b-ff75-4710-9661-2d7131817f6b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:44.641135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b650b5b-ff75-4710-9661-2d7131817f6b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:44.641135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b650b5b-ff75-4710-9661-2d7131817f6b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:44.642472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 548/1111] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:44.647593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:44.648004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:44.648146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:44.665573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:44.666739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:44.683238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:44.733632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 564/1112] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:44.757837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b650b5b-ff75-4710-9661-2d7131817f6b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:44.860005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b650b5b-ff75-4710-9661-2d7131817f6b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:44.894209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:44.894785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:44.895066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:44.895370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] PATH_INFO: `/v3/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:44.896752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] RBAC: Authorizing `identity:get_role(role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:44.898141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:44.899537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 549/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:44 2022] GET /identity/v3/roles/5a02bc464efd499bb3230455492613ae => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:44.905482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:45.002408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dbb6e-655c-4170-8764-7611cb9a0cba None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:45.034776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:45.035429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:45.035705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:45.035978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] PATH_INFO: `/v3/users/13954b6f7f8a4e3f9dfeb9a744303b0c` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:45.049240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=13954b6f7f8a4e3f9dfeb9a744303b0c)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:45.050705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:45.052439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 565/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:44 2022] GET /identity/v3/users/13954b6f7f8a4e3f9dfeb9a744303b0c => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:45.059340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:45.071486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:45.156111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb313745-2caf-45c2-aa2e-70ad064a4902 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:45.190441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:45.191042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:45.191346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:45.191618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:45.256092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:45.257540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:45.268229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:45.353494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 550/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:45.359926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:45.371936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:45.459434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a96b79-9eaa-44a3-b085-157811108bb9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:45.495809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c390e7-2ebf-4f22-a60c-5a639656f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:45.496453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c390e7-2ebf-4f22-a60c-5a639656f6da None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:45.496730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c390e7-2ebf-4f22-a60c-5a639656f6da None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:45.497053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c390e7-2ebf-4f22-a60c-5a639656f6da None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:45.498161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c390e7-2ebf-4f22-a60c-5a639656f6da None admin] RBAC: Authorizing `identity:get_project(project_id=5a644ac1066544598d7b3cef416933fb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:45.499617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c390e7-2ebf-4f22-a60c-5a639656f6da None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:45.501235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 566/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:45 2022] GET /identity/v3/projects/5a644ac1066544598d7b3cef416933fb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:45.507461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:45.604711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0da20-a311-476a-9de1-a06b3e6b259e None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:45.615592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:45.639942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:45.640547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:45.640834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:45.641366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:45.641819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] RBAC: Adding query filter params (role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=5a644ac1066544598d7b3cef416933fb, user.id=13954b6f7f8a4e3f9dfeb9a744303b0c) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:45.642168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:45.643600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02744786-74ed-4657-9dba-8ebbb54b5fc0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:45.648905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 551/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:45 2022] GET /identity/v3/role_assignments?user.id=13954b6f7f8a4e3f9dfeb9a744303b0c&scope.project.id=5a644ac1066544598d7b3cef416933fb&role.id=5a02bc464efd499bb3230455492613ae => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:47.624950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865e5e-aa27-4e14-96bc-c095258ff590 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:47.627022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865e5e-aa27-4e14-96bc-c095258ff590 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:47.627022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865e5e-aa27-4e14-96bc-c095258ff590 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:47.627347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 567/1118] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:47.632667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:47.633317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:47.633317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:47.651525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:47.652719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:47.719280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 552/1119] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:47.741597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99865e5e-aa27-4e14-96bc-c095258ff590 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:47.831253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:47.855627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99865e5e-aa27-4e14-96bc-c095258ff590 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:47.891459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:47.892094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:47.892374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:47.892649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] PATH_INFO: `/v3/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:47.894079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] RBAC: Authorizing `identity:get_role(role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:47.895502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:47.896848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 568/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:47 2022] GET /identity/v3/roles/5a02bc464efd499bb3230455492613ae => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:47.902885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:48.003699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6abee-6c38-4853-af94-114f70f85e3f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:48.006045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:48.038492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:48.039141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:48.039424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:48.039698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] PATH_INFO: `/v3/users/13954b6f7f8a4e3f9dfeb9a744303b0c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:48.042754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] RBAC: Authorizing `identity:get_user(user_id=13954b6f7f8a4e3f9dfeb9a744303b0c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:48.044251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:48.045851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 553/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:47 2022] GET /identity/v3/users/13954b6f7f8a4e3f9dfeb9a744303b0c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:48.051722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:48.151945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ccdf2-1c0e-44d5-8e51-228e0505fcb2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:48.186572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4798622-811d-414e-aaa1-a80be71acaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:48.187248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4798622-811d-414e-aaa1-a80be71acaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:48.187530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4798622-811d-414e-aaa1-a80be71acaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:48.187807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4798622-811d-414e-aaa1-a80be71acaa8 None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:48.188937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4798622-811d-414e-aaa1-a80be71acaa8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a644ac1066544598d7b3cef416933fb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:48.190374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4798622-811d-414e-aaa1-a80be71acaa8 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:48.192090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 569/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:48 2022] GET /identity/v3/projects/5a644ac1066544598d7b3cef416933fb => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:48.198210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:48.299052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a4a52-ca2f-4c84-ad43-b9f23b1f3996 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:48.333272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6719f-17ac-47a9-8050-bbc788af9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:48.333921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6719f-17ac-47a9-8050-bbc788af9c7b None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:48.334203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6719f-17ac-47a9-8050-bbc788af9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:48.334479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6719f-17ac-47a9-8050-bbc788af9c7b None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb/users/13954b6f7f8a4e3f9dfeb9a744303b0c/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:48.336618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6719f-17ac-47a9-8050-bbc788af9c7b None admin] RBAC: Authorizing `identity:create_grant(project_id=5a644ac1066544598d7b3cef416933fb, user_id=13954b6f7f8a4e3f9dfeb9a744303b0c, role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:48.338546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6719f-17ac-47a9-8050-bbc788af9c7b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:48.350910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 554/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:19:48 2022] PUT /identity/v3/projects/5a644ac1066544598d7b3cef416933fb/users/13954b6f7f8a4e3f9dfeb9a744303b0c/roles/5a02bc464efd499bb3230455492613ae => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:19:50.357184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad9e94-0ee9-40f3-9634-7be0ab585f07 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:50.357744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad9e94-0ee9-40f3-9634-7be0ab585f07 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:50.357744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ad9e94-0ee9-40f3-9634-7be0ab585f07 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:50.359063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 570/1124] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:50.364434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:50.364906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:50.365008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:50.383374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:50.385099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:50.414029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:50.457924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 555/1125] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:50.482106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad9e94-0ee9-40f3-9634-7be0ab585f07 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:50.498626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:50.586651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ad9e94-0ee9-40f3-9634-7be0ab585f07 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:50.620649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:50.621186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:50.621462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:50.621729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] PATH_INFO: `/v3/roles/5a02bc464efd499bb3230455492613ae` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:50.623136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] RBAC: Authorizing `identity:get_role(role_id=5a02bc464efd499bb3230455492613ae)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:50.624623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:50.625958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 571/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:50 2022] GET /identity/v3/roles/5a02bc464efd499bb3230455492613ae => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:50.632099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:50.734142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d26ee3-8ce6-4744-94a4-2976f4699b16 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:50.770291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:50.770920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:50.771226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:50.771501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] PATH_INFO: `/v3/users/13954b6f7f8a4e3f9dfeb9a744303b0c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:50.772618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] RBAC: Authorizing `identity:get_user(user_id=13954b6f7f8a4e3f9dfeb9a744303b0c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:50.774050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:50.775708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 556/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:50 2022] GET /identity/v3/users/13954b6f7f8a4e3f9dfeb9a744303b0c => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:50.781853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:50.868912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:50.888592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3b7c9-daa6-495a-8863-24d35f526770 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:50.924957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:50.925570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:50.925842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:50.926113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:50.995977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:50.997429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:51.078408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:51.097592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 572/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:51.104134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:51.134701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:51.207543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ffae51-5329-404c-b7a5-2e6b74754c3c None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:51.241109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfeb76-efe4-4dc7-a1f9-4b75ebc29f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:51.241743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfeb76-efe4-4dc7-a1f9-4b75ebc29f7c None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:51.242024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfeb76-efe4-4dc7-a1f9-4b75ebc29f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:51.242299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfeb76-efe4-4dc7-a1f9-4b75ebc29f7c None admin] PATH_INFO: `/v3/projects/5a644ac1066544598d7b3cef416933fb` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:51.243438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfeb76-efe4-4dc7-a1f9-4b75ebc29f7c None admin] RBAC: Authorizing `identity:get_project(project_id=5a644ac1066544598d7b3cef416933fb)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:51.245538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfeb76-efe4-4dc7-a1f9-4b75ebc29f7c None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:51.247209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 557/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:51 2022] GET /identity/v3/projects/5a644ac1066544598d7b3cef416933fb => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:51.253345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:51.360880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17b3a1-4e0e-41cc-b903-928024cdbc65 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:51.396074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:51.396599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:51.396882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:51.397157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:51.397888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] RBAC: Adding query filter params (role.id=5a02bc464efd499bb3230455492613ae, scope.project.id=5a644ac1066544598d7b3cef416933fb, user.id=13954b6f7f8a4e3f9dfeb9a744303b0c) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:51.398200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:51.399671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e000ac-dd7f-4665-91ea-419970a5c510 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:51.406342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 573/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:51 2022] GET /identity/v3/role_assignments?user.id=13954b6f7f8a4e3f9dfeb9a744303b0c&scope.project.id=5a644ac1066544598d7b3cef416933fb&role.id=5a02bc464efd499bb3230455492613ae => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:53.442690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a97e9-8cc4-4897-8431-71c3cd38e16b None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:53.443272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a97e9-8cc4-4897-8431-71c3cd38e16b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:53.443272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a97e9-8cc4-4897-8431-71c3cd38e16b None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:53.444639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 558/1131] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:53.451172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:53.451643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:53.451737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:53.469807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:53.470989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:53.512675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:53.536690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 574/1132] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:53.560129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a97e9-8cc4-4897-8431-71c3cd38e16b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:53.661147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:53.666552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a97e9-8cc4-4897-8431-71c3cd38e16b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:53.714169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e82f23-0cb2-4bab-a74c-787f4ffe3c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:53.715248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e82f23-0cb2-4bab-a74c-787f4ffe3c7d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:53.715728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e82f23-0cb2-4bab-a74c-787f4ffe3c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:53.716207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e82f23-0cb2-4bab-a74c-787f4ffe3c7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:53.717734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e82f23-0cb2-4bab-a74c-787f4ffe3c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:53.719232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e82f23-0cb2-4bab-a74c-787f4ffe3c7d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:53.720872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 559/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:53.726844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:53.826768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db02df-1f2f-4249-95a9-30095c562a36 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:53.862041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba7f2c-d4f3-4624-96a8-6ee22a47a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:53.862652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba7f2c-d4f3-4624-96a8-6ee22a47a508 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:53.862935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba7f2c-d4f3-4624-96a8-6ee22a47a508 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:53.863245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba7f2c-d4f3-4624-96a8-6ee22a47a508 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:53.863900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba7f2c-d4f3-4624-96a8-6ee22a47a508 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:53.865337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba7f2c-d4f3-4624-96a8-6ee22a47a508 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:53.893546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 575/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:19:53 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:55.939992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6e54-2f71-47b0-8c0a-621a5dd018a9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:55.940529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6e54-2f71-47b0-8c0a-621a5dd018a9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:55.940529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6e54-2f71-47b0-8c0a-621a5dd018a9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:55.941872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 560/1135] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:55.947352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:55.947810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:55.947907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:55.966227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:55.967435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:55.991150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:56.035956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 576/1136] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:56.061008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae6e54-2f71-47b0-8c0a-621a5dd018a9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:56.168674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae6e54-2f71-47b0-8c0a-621a5dd018a9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:56.202629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01384784-736d-4bda-8723-e4dab5d79233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:56.203290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01384784-736d-4bda-8723-e4dab5d79233 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:56.203570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01384784-736d-4bda-8723-e4dab5d79233 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:56.203840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01384784-736d-4bda-8723-e4dab5d79233 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:56.204951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01384784-736d-4bda-8723-e4dab5d79233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:56.206404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01384784-736d-4bda-8723-e4dab5d79233 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:56.208020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 561/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:19:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:56.214359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:56.325448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a2f9c-a4b4-4457-8609-97ffc162d857 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:56.361030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced7fbe-3d14-434c-8b72-095e6d3fd65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:56.361642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7fbe-3d14-434c-8b72-095e6d3fd65f None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:56.361915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7fbe-3d14-434c-8b72-095e6d3fd65f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:56.362183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced7fbe-3d14-434c-8b72-095e6d3fd65f None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:56.362828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7fbe-3d14-434c-8b72-095e6d3fd65f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:56.364297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ced7fbe-3d14-434c-8b72-095e6d3fd65f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:56.386266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 577/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:19:56 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:19:58.462197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68076d52-4df7-4f4b-9c8f-ee3d390f623e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:58.462747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68076d52-4df7-4f4b-9c8f-ee3d390f623e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:58.462873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68076d52-4df7-4f4b-9c8f-ee3d390f623e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:58.464171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 562/1139] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:19:58.469807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:58.470322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:58.470407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:58.489988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:19:58.491233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:19:58.502926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:58.568411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 578/1140] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:19:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:19:58.591743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68076d52-4df7-4f4b-9c8f-ee3d390f623e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:58.628222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:58.696418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68076d52-4df7-4f4b-9c8f-ee3d390f623e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:58.730517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:58.731055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:58.731363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:58.731633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:58.741831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:19:58.747051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:19:58.748247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:19:58.749160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 563/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:19:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:19:58.754744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:58.857660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768854b1-49c0-4eae-b9ae-aa2046107480 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:58.902105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:58.902749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:58.903032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:58.903337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:58.904026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:58.904339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:58.905780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:58.916105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 579/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:19:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:58.922277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:59.026917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:59.034505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204121a9-a227-4854-9350-36145c13fdd6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:59.079693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:59.080343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:59.080630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:59.080909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] PATH_INFO: `/v3/users/2cc2ef16b391471a8cd006fbbb688053` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:59.097386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] RBAC: Authorizing `identity:get_user(user_id=2cc2ef16b391471a8cd006fbbb688053)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:59.101702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:59.104897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 564/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:19:58 2022] GET /identity/v3/users/2cc2ef16b391471a8cd006fbbb688053 => generated 286 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:19:59.110675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:59.121216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:59.218163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b80d92-2b79-48f6-b90c-aec2bbcfd579 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:59.253215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:59.253831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:59.254108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:59.254376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:59.327753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:59.329216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:59.338003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:59.438979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 580/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:19:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:19:59.445312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:59.548996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e96a8-668f-4bfd-a635-9efd6711fbb0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:59.585939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9ed40-c67e-4521-b3d5-851786385950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:59.586482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9ed40-c67e-4521-b3d5-851786385950 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:59.586758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9ed40-c67e-4521-b3d5-851786385950 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:59.587026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9ed40-c67e-4521-b3d5-851786385950 None admin] PATH_INFO: `/v3/projects/03beb321d96049d18e5588d523f1f958` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:59.588182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9ed40-c67e-4521-b3d5-851786385950 None admin] RBAC: Authorizing `identity:get_project(project_id=03beb321d96049d18e5588d523f1f958)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:59.589621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9ed40-c67e-4521-b3d5-851786385950 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:59.591295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 565/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:19:59 2022] GET /identity/v3/projects/03beb321d96049d18e5588d523f1f958 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:19:59.597397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:19:59.706079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e4487f-9464-4f5a-9f09-b73c44342bae None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:19:59.715184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:19:59.746626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:19:59.747281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:19:59.747565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:19:59.747840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:19:59.748583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=03beb321d96049d18e5588d523f1f958, user.id=2cc2ef16b391471a8cd006fbbb688053) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:19:59.748895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:19:59.750618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025bf221-f25d-45ef-af3f-55362e86e9f3 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:19:59.767113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 581/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:19:59 2022] GET /identity/v3/role_assignments?user.id=2cc2ef16b391471a8cd006fbbb688053&scope.project.id=03beb321d96049d18e5588d523f1f958&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:01.818109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960e86-58d7-4130-bede-2fa59ae389ca None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:01.818666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960e86-58d7-4130-bede-2fa59ae389ca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:01.818666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa960e86-58d7-4130-bede-2fa59ae389ca None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:01.820008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 566/1147] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:01.826186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:01.826811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:01.826811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:01.846265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:01.847525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:01.920640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 582/1148] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:20:01.943505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa960e86-58d7-4130-bede-2fa59ae389ca None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:01.966021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:02.046851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa960e86-58d7-4130-bede-2fa59ae389ca None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:02.081184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:02.081816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:02.082095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:02.082364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:02.093441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:20:02.098088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:20:02.099187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:20:02.100161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 567/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:20:02.106116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:02.203622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:02.206828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e4c3a-10b2-42bc-8459-3a468a50a9d6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:02.241303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:02.241907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:02.242180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:02.242447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:02.243131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:02.243439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:02.244816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:02.253789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 583/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:20:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:02.261304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:02.344451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:02.365943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ae6f1b-1298-43a2-9341-334749009b37 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:02.401963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:02.402601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:02.402883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:02.403188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] PATH_INFO: `/v3/users/2cc2ef16b391471a8cd006fbbb688053` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:02.404327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] RBAC: Authorizing `identity:get_user(user_id=2cc2ef16b391471a8cd006fbbb688053)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:02.405804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:02.407446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 568/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:20:02 2022] GET /identity/v3/users/2cc2ef16b391471a8cd006fbbb688053 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:20:02.413257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:02.520397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082e65e-d450-41df-8468-d60e7ac64e99 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:02.553852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c318ed1-007d-4813-8c36-4d5b3e630851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:02.554484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318ed1-007d-4813-8c36-4d5b3e630851 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:02.554764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318ed1-007d-4813-8c36-4d5b3e630851 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:02.555035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c318ed1-007d-4813-8c36-4d5b3e630851 None admin] PATH_INFO: `/v3/projects/03beb321d96049d18e5588d523f1f958` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:02.557052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318ed1-007d-4813-8c36-4d5b3e630851 None admin] RBAC: Authorizing `identity:get_project(project_id=03beb321d96049d18e5588d523f1f958)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:02.558520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c318ed1-007d-4813-8c36-4d5b3e630851 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:02.560279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 584/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:20:02 2022] GET /identity/v3/projects/03beb321d96049d18e5588d523f1f958 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:02.569601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:02.677788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1963e-51e5-4966-9492-4c60a446cbd2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:02.712986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb29db-8c9f-400e-8646-134cc281fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:02.713646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb29db-8c9f-400e-8646-134cc281fbf5 None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:02.713921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb29db-8c9f-400e-8646-134cc281fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:02.714197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb29db-8c9f-400e-8646-134cc281fbf5 None admin] PATH_INFO: `/v3/projects/03beb321d96049d18e5588d523f1f958/users/2cc2ef16b391471a8cd006fbbb688053/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:02.716156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb29db-8c9f-400e-8646-134cc281fbf5 None admin] RBAC: Authorizing `identity:create_grant(project_id=03beb321d96049d18e5588d523f1f958, user_id=2cc2ef16b391471a8cd006fbbb688053, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:02.717591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb29db-8c9f-400e-8646-134cc281fbf5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:02.731865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 569/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:20:02 2022] PUT /identity/v3/projects/03beb321d96049d18e5588d523f1f958/users/2cc2ef16b391471a8cd006fbbb688053/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:20:04.760703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2663455-6831-441d-a6c0-40e270b098d6 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:04.761289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2663455-6831-441d-a6c0-40e270b098d6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:04.761289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2663455-6831-441d-a6c0-40e270b098d6 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:04.762567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 585/1154] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:04.768403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:04.769018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:04.769018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:04.787765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:04.788960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:04.791313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:04.860874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 570/1155] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:20:04.884651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2663455-6831-441d-a6c0-40e270b098d6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:04.916954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:04.989112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2663455-6831-441d-a6c0-40e270b098d6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:05.023157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:05.023704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:05.023984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:05.024259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:05.035016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:20:05.039654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:20:05.040766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:05.041825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:20:05.044857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 586/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:20:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:20:05.051163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:05.167517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f26beb-4160-4ec6-90ad-33fc6c112f93 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:05.201845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:05.202447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:05.202726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:05.202998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:05.203688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:05.203991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:05.205387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:05.210603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:05.214527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 571/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:20:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:05.220206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:05.311772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:05.326069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d28c-b562-4a96-91cf-8acfc18a4f3d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:05.360012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:05.360641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:05.360919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:05.361191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] PATH_INFO: `/v3/users/2cc2ef16b391471a8cd006fbbb688053` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:05.362315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] RBAC: Authorizing `identity:get_user(user_id=2cc2ef16b391471a8cd006fbbb688053)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:05.363812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:05.365404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 587/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:20:05 2022] GET /identity/v3/users/2cc2ef16b391471a8cd006fbbb688053 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:05.372833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:05.479575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412151bb-d4aa-487b-b6ac-44ca1a4f16b4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:05.513986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:05.514682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:05.514966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:05.515273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:05.582832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:05.585175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:05.587921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:05.698724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 572/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:20:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 14:20:05.704736 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:05.824008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b093-ae65-412e-942b-e3ac08dd8800 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:05.860161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908500-148c-4543-8748-390fdc3e7845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:05.860719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908500-148c-4543-8748-390fdc3e7845 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:05.861003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908500-148c-4543-8748-390fdc3e7845 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:05.861275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908500-148c-4543-8748-390fdc3e7845 None admin] PATH_INFO: `/v3/projects/03beb321d96049d18e5588d523f1f958` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:05.862427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908500-148c-4543-8748-390fdc3e7845 None admin] RBAC: Authorizing `identity:get_project(project_id=03beb321d96049d18e5588d523f1f958)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:05.864066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908500-148c-4543-8748-390fdc3e7845 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:05.865687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 588/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:20:05 2022] GET /identity/v3/projects/03beb321d96049d18e5588d523f1f958 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:05.871966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:05.974523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:05.977757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50103c89-fbc9-4e7a-b08b-5d477183f4ac None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:06.013020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:06.013634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:06.013913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:06.014194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:06.014921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=03beb321d96049d18e5588d523f1f958, user.id=2cc2ef16b391471a8cd006fbbb688053) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:06.015334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:06.016781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbf0e5-2705-4b4c-987e-0209501d4c37 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:06.022760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 573/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:20:05 2022] GET /identity/v3/role_assignments?user.id=2cc2ef16b391471a8cd006fbbb688053&scope.project.id=03beb321d96049d18e5588d523f1f958&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:08.100872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91407b5-50d8-4b5c-8316-a8597075696d None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:08.101493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91407b5-50d8-4b5c-8316-a8597075696d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:08.101493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91407b5-50d8-4b5c-8316-a8597075696d None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:08.103345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 589/1162] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:08.109644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:08.110403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:08.110403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:08.130274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:08.131486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:08.205516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 574/1163] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:20:08.228623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91407b5-50d8-4b5c-8316-a8597075696d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:08.261614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:08.343886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91407b5-50d8-4b5c-8316-a8597075696d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:08.377878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:08.378492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:08.378767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:08.379035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:08.449919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:08.451322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:08.478936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:08.556163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 590/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:20:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:20:08.562156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:08.657549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:08.664989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fd066-6443-4931-b1db-82de690399b5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:08.701982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:08.702530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:08.702905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:08.703212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:08.710621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:08.712112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-f38b42f5-a242-4764-bd60-52c0a7fa564d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:20:08.724373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:20:08.725708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:20:08.726666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 575/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:20:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:20:08.734234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:08.873332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1111-6353-4f0c-a7f4-589b7fe38d3c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:08.905379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:08.916674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:08.917703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:08.918175 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:08.918636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:08.919844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:08.920377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:08.922935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37eb74-2b29-4464-9373-a949558f2520 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:08.936460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 591/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Mon Aug 8 14:20:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:11.026726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe566-491f-429b-8763-e1945315db3c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:11.027297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe566-491f-429b-8763-e1945315db3c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:11.027421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742fe566-491f-429b-8763-e1945315db3c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:11.028709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 576/1167] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:11.036545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b57973-8ec8-42f2-b379-6f7fd63953cc None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:11.037122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b57973-8ec8-42f2-b379-6f7fd63953cc None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:11.037122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b57973-8ec8-42f2-b379-6f7fd63953cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:11.057492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e8b57973-8ec8-42f2-b379-6f7fd63953cc None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:11.058746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e8b57973-8ec8-42f2-b379-6f7fd63953cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:11.137101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 592/1168] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:20:11.160721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe566-491f-429b-8763-e1945315db3c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:11.269841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742fe566-491f-429b-8763-e1945315db3c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:11.307211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56ad79-7b0d-40f8-8905-808d518e1816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:11.307826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56ad79-7b0d-40f8-8905-808d518e1816 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:11.308105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56ad79-7b0d-40f8-8905-808d518e1816 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:11.308377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56ad79-7b0d-40f8-8905-808d518e1816 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:11.309059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56ad79-7b0d-40f8-8905-808d518e1816 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:11.310432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a56ad79-7b0d-40f8-8905-808d518e1816 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:11.328599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 577/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Mon Aug 8 14:20:11 2022] POST /identity/v3/domains => generated 267 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:20:13.435687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bac13-f373-4329-8b20-e43e301f815c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:13.436252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bac13-f373-4329-8b20-e43e301f815c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:13.436252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bac13-f373-4329-8b20-e43e301f815c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:13.437625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 593/1170] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:13.444499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:13.445035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:13.445486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:13.473671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:13.475824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:13.495112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:13.574566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 578/1171] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:20:13.598575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bac13-f373-4329-8b20-e43e301f815c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:13.714232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bac13-f373-4329-8b20-e43e301f815c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:13.738345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:13.755035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4470efe-ffac-4a5d-ac9b-682a300d2faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:13.755729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4470efe-ffac-4a5d-ac9b-682a300d2faa None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:13.756011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4470efe-ffac-4a5d-ac9b-682a300d2faa None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:13.756289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4470efe-ffac-4a5d-ac9b-682a300d2faa None admin] PATH_INFO: `/v3/domains/7a441a8140f14876916864b0b9bc75ac` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:13.768445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4470efe-ffac-4a5d-ac9b-682a300d2faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=7a441a8140f14876916864b0b9bc75ac)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:13.769930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4470efe-ffac-4a5d-ac9b-682a300d2faa None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:13.771704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 594/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Mon Aug 8 14:20:13 2022] GET /identity/v3/domains/7a441a8140f14876916864b0b9bc75ac => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:20:13.778294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:13.890381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768135af-32e3-4b50-bb10-edc9fbc522fa None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:13.932293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbab8bf-998b-4fd8-9c5c-64ebb95d17fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:13.932924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbab8bf-998b-4fd8-9c5c-64ebb95d17fe None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:13.933203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbab8bf-998b-4fd8-9c5c-64ebb95d17fe None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:13.933476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbab8bf-998b-4fd8-9c5c-64ebb95d17fe None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:13.934138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbab8bf-998b-4fd8-9c5c-64ebb95d17fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:13.935702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbab8bf-998b-4fd8-9c5c-64ebb95d17fe None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:13.952268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:13.967493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 579/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:20:13 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:20:16.107482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32390a6-e84a-4c4f-9d73-95bcffd7b336 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:16.108027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32390a6-e84a-4c4f-9d73-95bcffd7b336 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:16.108027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32390a6-e84a-4c4f-9d73-95bcffd7b336 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:16.109630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 595/1174] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:16.115628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:16.116178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:16.116271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:16.135791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:16.136932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:16.210548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 580/1175] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:20:16.236857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32390a6-e84a-4c4f-9d73-95bcffd7b336 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:16.354585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32390a6-e84a-4c4f-9d73-95bcffd7b336 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:16.394009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e3c3a-853c-408d-85dd-ecddf09a5026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:16.394565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3c3a-853c-408d-85dd-ecddf09a5026 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:16.394841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3c3a-853c-408d-85dd-ecddf09a5026 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:16.395322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3c3a-853c-408d-85dd-ecddf09a5026 None admin] PATH_INFO: `/v3/domains/7a441a8140f14876916864b0b9bc75ac` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:16.396616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e3c3a-853c-408d-85dd-ecddf09a5026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7a441a8140f14876916864b0b9bc75ac)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:16.398110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e3c3a-853c-408d-85dd-ecddf09a5026 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:16.400480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 596/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Mon Aug 8 14:20:16 2022] GET /identity/v3/domains/7a441a8140f14876916864b0b9bc75ac => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:16.406780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:16.520829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f666c81-1829-4d9b-9357-28631e4071a3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:16.550216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:16.559659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbbce8-7ad4-4648-a716-5da239bc5a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:16.560275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbbce8-7ad4-4648-a716-5da239bc5a14 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:16.560552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbbce8-7ad4-4648-a716-5da239bc5a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:16.560821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbbce8-7ad4-4648-a716-5da239bc5a14 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:16.561468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbbce8-7ad4-4648-a716-5da239bc5a14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:16.562910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbbce8-7ad4-4648-a716-5da239bc5a14 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:16.586949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 581/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:20:16 2022] POST /identity/v3/users => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:20:18.694632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d3ed-a368-4e81-986d-8cebdc367b43 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:18.695500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d3ed-a368-4e81-986d-8cebdc367b43 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:18.695500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d3ed-a368-4e81-986d-8cebdc367b43 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:18.696544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 597/1178] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:20:18.702364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:18.703171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:18.703171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:18.721942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:18.723201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:18.805968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 582/1179] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:20:18.830183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d3ed-a368-4e81-986d-8cebdc367b43 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:18.901279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:18.972437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d3ed-a368-4e81-986d-8cebdc367b43 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:19.016274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:19.016915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:19.017189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:19.017460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:19.029773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:20:19.034480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:20:19.035708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:20:19.036910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 598/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:20:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:20:19.045584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:19.221825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deef406-8dce-4e55-bf40-23b4edb6963f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:19.238724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:19.266082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:19.266717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:19.267004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:19.267387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:19.268096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:19.268405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:19.269821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:19.289919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 583/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:20:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:19.296126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:19.403001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f6d61-d41f-4bb2-8e47-d18ab645f1a2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:19.412169 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:19.437525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:19.438156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:19.438435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:19.438705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] PATH_INFO: `/v3/users/d12e048289864064a61da40c06931427` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:19.456092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=d12e048289864064a61da40c06931427)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:19.458856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:19.462697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 599/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:20:19 2022] GET /identity/v3/users/d12e048289864064a61da40c06931427 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:19.468441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:19.567481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670757c6-1c36-440d-9d5b-6e91a99a96b7 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:19.601961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:19.602583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:19.602865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:19.603162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:19.645820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:19.672794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:19.674275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:19.788345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 584/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:20:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:20:19.794285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:19.903657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d819c-fb5c-45d7-a8b4-66f9a2e41bb2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:19.938803 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bc089-82e5-486f-97ff-0f5fd70dde63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:19.939461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc089-82e5-486f-97ff-0f5fd70dde63 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:19.939740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc089-82e5-486f-97ff-0f5fd70dde63 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:19.940008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bc089-82e5-486f-97ff-0f5fd70dde63 None admin] PATH_INFO: `/v3/projects/a0e03082352f458ba04d8159f11b3df8` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:19.941140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc089-82e5-486f-97ff-0f5fd70dde63 None admin] RBAC: Authorizing `identity:get_project(project_id=a0e03082352f458ba04d8159f11b3df8)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:19.942554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bc089-82e5-486f-97ff-0f5fd70dde63 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:19.944254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 600/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:20:19 2022] GET /identity/v3/projects/a0e03082352f458ba04d8159f11b3df8 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:19.951054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:20.024543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:20.050268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638131a9-051c-44ed-a15b-257cdad49792 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:20.086805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:20.087514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:20.087834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:20.088147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:20.088939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=a0e03082352f458ba04d8159f11b3df8, user.id=d12e048289864064a61da40c06931427) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:20.089296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:20.090842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a97a5-5192-4f39-8695-b0dd4c84c23d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:20.097310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 585/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:20:19 2022] GET /identity/v3/role_assignments?user.id=d12e048289864064a61da40c06931427&scope.project.id=a0e03082352f458ba04d8159f11b3df8&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:22.107234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b570c-7387-4a9e-9126-0433439cd4d4 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.107826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b570c-7387-4a9e-9126-0433439cd4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.107826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b570c-7387-4a9e-9126-0433439cd4d4 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.109111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 601/1186] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:22.114346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.114749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.114896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.133323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:22.134447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:22.201673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 586/1187] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:20:22.223959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b570c-7387-4a9e-9126-0433439cd4d4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:22.258764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:22.324113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b570c-7387-4a9e-9126-0433439cd4d4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:22.358936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:22.359576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.359850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.360118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.369916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:20:22.374476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:20:22.375611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:20:22.376681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 602/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:20:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:20:22.381607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:22.447412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:22.480105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60903cc3-4dcf-4189-918b-0b7b581fff55 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:22.512808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:22.513412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.513690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.513961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.514616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:22.514917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:22.516304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:22.524762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 587/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:20:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:20:22.530641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:22.621071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:22.628100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c9f75-2bce-496d-9ac9-a3f1ad124862 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:22.661535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:22.662159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.662438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.662709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] PATH_INFO: `/v3/users/d12e048289864064a61da40c06931427` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.663845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] RBAC: Authorizing `identity:get_user(user_id=d12e048289864064a61da40c06931427)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:22.665320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:22.666915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 603/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:20:22 2022] GET /identity/v3/users/d12e048289864064a61da40c06931427 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:22.672796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:22.775550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f218a0-11a7-413f-af7a-73352b5c3a83 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:22.804133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:22.810960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f988d8-f326-44f6-b153-fc13476ffe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:22.811633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f988d8-f326-44f6-b153-fc13476ffe96 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.811915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f988d8-f326-44f6-b153-fc13476ffe96 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.812191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f988d8-f326-44f6-b153-fc13476ffe96 None admin] PATH_INFO: `/v3/projects/a0e03082352f458ba04d8159f11b3df8` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.813326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f988d8-f326-44f6-b153-fc13476ffe96 None admin] RBAC: Authorizing `identity:get_project(project_id=a0e03082352f458ba04d8159f11b3df8)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:22.814771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f988d8-f326-44f6-b153-fc13476ffe96 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:22.816486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 588/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:20:22 2022] GET /identity/v3/projects/a0e03082352f458ba04d8159f11b3df8 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:20:22.823136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:22.921682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5951a54-433c-41a5-9881-0bbfe71e1772 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:22.954745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db081af-738a-428b-9b8c-30b3cd8ab254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:22.955418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db081af-738a-428b-9b8c-30b3cd8ab254 None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:22.955696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db081af-738a-428b-9b8c-30b3cd8ab254 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:22.955966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db081af-738a-428b-9b8c-30b3cd8ab254 None admin] PATH_INFO: `/v3/projects/a0e03082352f458ba04d8159f11b3df8/users/d12e048289864064a61da40c06931427/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:22.957864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db081af-738a-428b-9b8c-30b3cd8ab254 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0e03082352f458ba04d8159f11b3df8, user_id=d12e048289864064a61da40c06931427, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:22.959316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db081af-738a-428b-9b8c-30b3cd8ab254 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:22.971765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 604/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:20:22 2022] PUT /identity/v3/projects/a0e03082352f458ba04d8159f11b3df8/users/d12e048289864064a61da40c06931427/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:20:24.935773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec83a1-329b-43f9-96e6-a007d89fa45e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:24.936308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec83a1-329b-43f9-96e6-a007d89fa45e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:24.936308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec83a1-329b-43f9-96e6-a007d89fa45e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:24.937660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 589/1193] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:20:24.942602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:24.943003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:24.943183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:24.960781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:20:24.961974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:20:24.978396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:25.028857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 605/1194] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:20:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:20:25.052246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec83a1-329b-43f9-96e6-a007d89fa45e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:25.157119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec83a1-329b-43f9-96e6-a007d89fa45e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:25.190605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:25.191258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:25.191535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:25.191804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:25.202028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:20:25.206585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:20:25.207709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:20:25.214127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 590/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:20:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:20:25.214998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:25.311486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d582d4-4aa2-4802-976c-8c4a8cbfd491 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:25.344594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:25.345192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:25.345479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:25.345748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:25.346404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:25.346714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:25.348118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:25.356658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 606/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:20:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:25.362608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:25.420330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:25.462787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99376a17-cf5b-42de-af24-9877d150ae38 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:25.496595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1326942-5597-46c2-8292-140795d01080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:25.497230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1326942-5597-46c2-8292-140795d01080 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:25.497508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1326942-5597-46c2-8292-140795d01080 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:25.497782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1326942-5597-46c2-8292-140795d01080 None admin] PATH_INFO: `/v3/users/d12e048289864064a61da40c06931427` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:25.498908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1326942-5597-46c2-8292-140795d01080 None admin] RBAC: Authorizing `identity:get_user(user_id=d12e048289864064a61da40c06931427)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:25.500391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1326942-5597-46c2-8292-140795d01080 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:25.501971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 591/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Mon Aug 8 14:20:25 2022] GET /identity/v3/users/d12e048289864064a61da40c06931427 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:25.508347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:25.524818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:25.606830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee2141-62b8-422a-8ee8-d4229c2f7abc None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:25.640346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:25.640954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:25.641230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:25.641499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:25.708464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:25.709824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:25.725461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:25.820219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 607/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:20:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:20:25.826959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1326942-5597-46c2-8292-140795d01080 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:25.927600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1326942-5597-46c2-8292-140795d01080 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:25.955459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:25.964789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f4cec-1065-4cfb-9f0b-980fb1f91ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:25.965361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f4cec-1065-4cfb-9f0b-980fb1f91ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:25.965640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f4cec-1065-4cfb-9f0b-980fb1f91ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:25.965917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f4cec-1065-4cfb-9f0b-980fb1f91ec7 None admin] PATH_INFO: `/v3/projects/a0e03082352f458ba04d8159f11b3df8` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:25.967038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f4cec-1065-4cfb-9f0b-980fb1f91ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=a0e03082352f458ba04d8159f11b3df8)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:25.968495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f4cec-1065-4cfb-9f0b-980fb1f91ec7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:25.970136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 592/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Mon Aug 8 14:20:25 2022] GET /identity/v3/projects/a0e03082352f458ba04d8159f11b3df8 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:25.976236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:20:26.071477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7c8e9-94ad-4170-a5a2-4b8c6b9fc210 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:20:26.085055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:26.105235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:20:26.105830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:20:26.106110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:20:26.106378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:20:26.107108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=a0e03082352f458ba04d8159f11b3df8, user.id=d12e048289864064a61da40c06931427) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:20:26.107416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:20:26.108819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d1055-df87-42fb-bd5c-7208bfe89ef5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:20:26.114422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 608/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:20:25 2022] GET /identity/v3/role_assignments?user.id=d12e048289864064a61da40c06931427&scope.project.id=a0e03082352f458ba04d8159f11b3df8&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:20:35.960944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:20:36.111902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=11 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:20.993139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e7d50-b712-4467-b5df-3818252c0960 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:20.993747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e7d50-b712-4467-b5df-3818252c0960 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:20.993747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e7d50-b712-4467-b5df-3818252c0960 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:20.995368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 593/1201] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:21.001428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:21.001957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:21.002089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:21.022960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:21.024363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:21.129140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 609/1202] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 14:21:21.154945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e7d50-b712-4467-b5df-3818252c0960 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:21.306985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e7d50-b712-4467-b5df-3818252c0960 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:21.344639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:21.345255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:21.345541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:21.345819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:21.358216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:21.359730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:21.375037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:21.376225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:21.377384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:21.377384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:21.377384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:21.381374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 594/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:21.382294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:21.522684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6d67-8981-4eaa-939b-9f4786af19d6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:21.558561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:21.566073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:21.566791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:21.567114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:21.567464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:21.568252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:21.568609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:21.570225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de4174-a5fc-4c7f-bcee-9d5fc85f9bd2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:21.586288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 610/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:21:21.592042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:21.660138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:21.716749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631da70-9aff-42f3-a81d-8a40b5146287 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:21.757465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56bd75-58e5-469e-856f-4a94f095f5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:21.758122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56bd75-58e5-469e-856f-4a94f095f5a2 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:21.758430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56bd75-58e5-469e-856f-4a94f095f5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:21.758722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56bd75-58e5-469e-856f-4a94f095f5a2 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:21.759470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56bd75-58e5-469e-856f-4a94f095f5a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:21.761069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56bd75-58e5-469e-856f-4a94f095f5a2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:21.796065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 595/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Mon Aug 8 14:21:21 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:21:24.220448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96809546-88b8-47f7-b34a-091ade49c017 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:24.221021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96809546-88b8-47f7-b34a-091ade49c017 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:24.221021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96809546-88b8-47f7-b34a-091ade49c017 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:24.222416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 611/1206] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:21:24.228509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:24.228973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:24.229060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:24.250080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:24.251658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:24.335827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 596/1207] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:24.360468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96809546-88b8-47f7-b34a-091ade49c017 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:24.493917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96809546-88b8-47f7-b34a-091ade49c017 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:24.532410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:24.533111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:24.533401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:24.533681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:24.546009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:21:24.550932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:24.552018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:21:24.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:24.559746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 612/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:21:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:24.560870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:24.606413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:24.684937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c263b42-e1ed-48a6-a3d6-5ef52b02bfc4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:24.723926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:24.724726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:24.725104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:24.725495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:24.726274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:24.726680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:24.728191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:24.737188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 597/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:21:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:24.743308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:24.801911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:24.873560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e8a9a-fffd-4806-8c74-54d254ac3978 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:24.919587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:24.920256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:24.920539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:24.920839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:24.933731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:24.935394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:24.950772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:24.952032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:24.953206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:24.953206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:24.953206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:24.953206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 613/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:24.957613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:25.082740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:25.086388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac966486-bcbe-453c-abee-b16ab25504b2 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:25.128572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:25.129589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:25.130451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:25.130451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:25.131237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:25.131546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:25.133067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:25.146827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 598/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:25.153092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:25.264146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300e003-0681-408e-a796-f1f9b7ddf010 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:25.265778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:25.302504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:25.303433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:25.303759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:25.304030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:25.312585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:25.314123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:21:25.326323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:25.327513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:25.328757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 614/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:25.335537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:25.447461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53710ac-e31b-4a21-b41f-08122f1e53b5 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:25.494853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:25.495609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:25.495951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:25.496260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:25.497002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:25.497310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:25.498755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:25.524027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 599/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 14:21:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:25.530290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:25.650527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f26844-48dd-4a8a-a492-c13e9461ea9e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:25.695462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:25.696080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:25.696418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:25.696699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:25.734789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:25.790457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:25.792431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:25.938612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 615/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:21:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:21:25.946647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:25.968261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:26.076341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a016c3-ad83-4318-9171-4c0462fa5b88 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:26.113850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:26.114996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:26.115302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:26.115606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:26.127350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:26.129067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:26.148632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:26.149797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:26.150686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:26.150686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:26.150686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:26.150686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 600/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:26.155843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:26.224492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:26.266410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ed04e-4e59-4242-b2ae-a42df6a2fd06 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:26.303542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:26.304145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:26.304448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:26.304679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:26.305349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:26.305652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:26.307016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:26.324448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 616/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:26.326239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:26.414202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:26.453486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535b1e5-4470-468c-b46e-cc09eb5754d0 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:26.497237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:26.498158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:26.498397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:26.498668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:26.506393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:26.507893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:21:26.518938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:26.524914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:26.526024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 601/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:26.526641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:26.643650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58241a07-9d0c-4bd7-83a8-edca5a60ae77 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:26.668501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:26.683661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:26.684412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:26.684794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:26.685159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:26.685995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:26.686414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:26.688113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd076a-8faf-45d4-a73b-1178ef075931 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:26.704101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 617/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:26.710270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:26.843308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09606e39-e605-4b10-8039-9f880ae6def6 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:26.885265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:26.894006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:26.894704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:26.895077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:26.895458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:26.896327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=90fd9b3148004cc2b8d14bced329b10b) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:26.896770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:26.898734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e006ce-aefb-4e8f-a651-7cd14dd197b4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:26.904813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 602/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:21:26 2022] GET /identity/v3/role_assignments?user.id=90fd9b3148004cc2b8d14bced329b10b&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:29.053627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb57aa-6c15-4d7f-bfa7-33fe67ea4cdb None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:29.054338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb57aa-6c15-4d7f-bfa7-33fe67ea4cdb None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:29.054338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb57aa-6c15-4d7f-bfa7-33fe67ea4cdb None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:29.055741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 618/1220] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:29.062602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:29.063123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:29.063427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:29.086053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:29.087420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:29.163788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 603/1221] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:29.187481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb57aa-6c15-4d7f-bfa7-33fe67ea4cdb None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:29.297170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb57aa-6c15-4d7f-bfa7-33fe67ea4cdb None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:29.334801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:29.335457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:29.335730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:29.335996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:29.346100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:21:29.350773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:29.351926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:21:29.352985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:29.355317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 619/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:21:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:29.358059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:29.469209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85438-34fd-43af-9974-8ca22f890757 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:29.499013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:29.512641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:29.513376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:29.513661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:29.513943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:29.514635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] RBAC: Adding query filter params (name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:29.514948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:29.516458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:29.530039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 604/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:21:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:21:29.531468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:29.578311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:29.639774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7c816-6ae0-482f-b6cb-73564e177652 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:29.677055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:29.678129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:29.678461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:29.678723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:29.689797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:29.691276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:29.709013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:29.710235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:29.711386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:29.711386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:29.711386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:29.711386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 620/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:29.718074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:29.827734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d6f85-554e-41e2-8bd4-8ddd63f7f384 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:29.865445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:29.866052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:29.866329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:29.866599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:29.867272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:29.867575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:29.868951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:29.883862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 605/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:29.890478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:29.998740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:30.006005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f14b7-aaea-43cb-b48f-2494bae1dce0 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:30.039901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:30.040533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:30.040811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:30.041089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:30.048717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:30.050158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:21:30.061456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:30.068506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:30.069630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 621/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:30.070316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:30.125101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:30.184549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec117ff-48fa-4c4a-a91f-835ee461c030 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:30.221135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:30.221791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:30.222113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:30.222402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:30.223253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:30.223676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:30.225345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:30.244633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 606/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 14:21:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:21:30.251029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:30.360025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6deea-1001-406b-8ae8-7b42e5942a49 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:30.399456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:30.400010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:30.400286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:30.400559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:30.412369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:30.413832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:30.429261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:30.430416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:30.431557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:30.431557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:30.431557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:30.431557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 622/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:21:30.436655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:30.552129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4d8c1-7fb4-4e83-b1cc-95520e097562 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:30.554754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:30.590220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:30.591003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:30.591284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:30.591533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:30.592176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:30.592484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:30.593854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:30.607747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 607/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:21:30.616691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:30.626337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:30.738092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ec8a4-707a-4fe5-a9a2-5b05ee18e646 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:30.773621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:30.774292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:30.774616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:30.774892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:30.782625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:30.784238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:21:30.800223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:30.801443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:30.802346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 623/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:30.808042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:30.925152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06a9a2-dbba-467c-8b2e-3861c801119a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:30.962775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:30.963519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:30.963801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:30.964071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:30.964756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:30.965059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:30.966478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77add246-f125-4981-bae2-270ea72553d4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:30.979603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 608/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:21:30.986142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:31.060096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:31.100254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78921a-0520-4b09-8f73-cb9911c2d240 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:31.137129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d39d83-318d-4dca-bc18-1d1ac1445b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:31.137785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d39d83-318d-4dca-bc18-1d1ac1445b4d None admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:31.138063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d39d83-318d-4dca-bc18-1d1ac1445b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:31.138341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d39d83-318d-4dca-bc18-1d1ac1445b4d None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/90fd9b3148004cc2b8d14bced329b10b/roles/8b7383b636eb4abba506f16a432e1f58` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:31.153690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d39d83-318d-4dca-bc18-1d1ac1445b4d None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=90fd9b3148004cc2b8d14bced329b10b, role_id=8b7383b636eb4abba506f16a432e1f58)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:31.155217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d39d83-318d-4dca-bc18-1d1ac1445b4d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:31.170203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 624/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:21:30 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/90fd9b3148004cc2b8d14bced329b10b/roles/8b7383b636eb4abba506f16a432e1f58 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:21:33.245983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1bb12-01d8-4602-a1a7-3223e2a6e5a7 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:33.246662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1bb12-01d8-4602-a1a7-3223e2a6e5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:33.246662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1bb12-01d8-4602-a1a7-3223e2a6e5a7 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:33.248077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 609/1233] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:33.253712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:33.254220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:33.254836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:33.272940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:33.274128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:33.346614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 625/1234] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:33.375340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1bb12-01d8-4602-a1a7-3223e2a6e5a7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:33.406396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:33.488214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1bb12-01d8-4602-a1a7-3223e2a6e5a7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:33.528463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:33.529159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:33.529546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:33.529819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:33.541263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:21:33.546083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:33.547021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:33.548108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 610/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Mon Aug 8 14:21:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:33.555139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:33.604094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:33.663003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1ed98-7ca2-43ea-b0cc-516688e88ca1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:33.705940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:33.706869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:33.707179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:33.707481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:33.708510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:33.708940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:33.710708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:33.720737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 626/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Mon Aug 8 14:21:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:33.728219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:33.820113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:33.837140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4426a57-f7af-452d-843f-c6df7be9a361 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:33.879791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:33.880503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:33.880791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:33.881070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:33.891753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:33.893224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:33.908879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:33.910020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:33.911496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:33.911496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:33.911496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:33.911496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 611/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:33.916940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:34.022393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace4258-b1d6-4da2-acd9-bbf2aa998e88 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:34.031580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:34.057910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:34.058540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:34.058806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:34.059067 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:34.059745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:34.060072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:34.061416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:34.073719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 627/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:34.079559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:34.193389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ea3dd-601f-4db8-8a07-3be528c00b42 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:34.236321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:34.236959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:34.237245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:34.237521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:34.240568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:34.247218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:34.249026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:21:34.264738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:34.266684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:34.267673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 612/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:34.272406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:34.387268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76066b-b70f-47db-9c7b-dcdbc420a05b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:34.425585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:34.426226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:34.426708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:34.427220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:34.428195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:34.428510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:34.430113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:34.451131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 628/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 14:21:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:34.457287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:34.586832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee3d64-abe8-43c7-87f5-2d1827dd5886 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:34.626684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:34.627398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:34.627704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:34.627956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:34.698788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:34.706609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:34.708663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:34.830139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 613/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:21:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:21:34.836396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:34.867902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:34.948082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7608ff9-6d9c-4b8e-9660-08cb70483a2f None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:34.983789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:34.984482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:34.984829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:34.985157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:34.997086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:34.998722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:35.015919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:35.017337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:35.018528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:35.018528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:35.018528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:35.018528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 629/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:35.023495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:35.131461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:35.138700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da720fc-bd24-401d-8dde-b977497121fa None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:35.179431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:35.180041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:35.180317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:35.180591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:35.181226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:35.181690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:35.183408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:35.201626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 614/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:35.203421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:35.297205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:35.324374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee9fa5-6148-405b-8fb0-14f1e6abd9e1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:35.362176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:35.362907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:35.363280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:35.364130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:35.375434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:35.377142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:21:35.390969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:35.392154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:35.393390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 630/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:35.398515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:35.524551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996a42-c852-456e-ba9a-9c2162b757bc None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:35.560098 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:35.561143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:35.561473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:35.561769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:35.562687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:35.563028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:35.564790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269baff-b9a5-4d2a-93fc-8f04a05ccd40 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:35.579413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 615/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:35.587185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:35.700660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab9e3d-982b-460e-be03-791470e7d28e None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:35.738683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:35.739377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:35.739659 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:35.739993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:35.740737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] RBAC: Adding query filter params (role.id=8b7383b636eb4abba506f16a432e1f58, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=90fd9b3148004cc2b8d14bced329b10b) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:35.741120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:35.742691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ddce6-7c39-48f0-924f-da6b9e038fde None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:35.748521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 631/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:21:35 2022] GET /identity/v3/role_assignments?user.id=90fd9b3148004cc2b8d14bced329b10b&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=8b7383b636eb4abba506f16a432e1f58 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:38.008591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3dcab-6de0-48d2-b9a4-1e2ed1736a92 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:38.009263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3dcab-6de0-48d2-b9a4-1e2ed1736a92 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:38.009263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3dcab-6de0-48d2-b9a4-1e2ed1736a92 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:38.010485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 616/1247] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:38.015989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:38.016461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:38.016576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:38.023249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:38.037033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:38.038386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:38.118751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 632/1248] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:38.143435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3dcab-6de0-48d2-b9a4-1e2ed1736a92 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:38.165292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:38.257012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3dcab-6de0-48d2-b9a4-1e2ed1736a92 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:38.296590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:38.297301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:38.297650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:38.298016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:38.311998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:21:38.317480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:38.318744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:38.319811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 617/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:38.327831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:38.459320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c54ec-9174-4c0d-a6d7-4ce6e231a50c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:38.504942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:38.512841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:38.513654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:38.514010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:38.514462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:38.515307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:38.515684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:38.517619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:38.530505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 633/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:38.537919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:38.638936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:38.666847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b1852-cb2e-48da-8ca9-f29fcfea0f14 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:38.707836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:38.708597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:38.708852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:38.709141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:38.720597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:38.721988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:38.738001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:38.739235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:38.740150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:38.740150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:38.740150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:38.740150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 618/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:38.745404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:38.869244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1c36-97ba-4a3c-a100-ed7db337b826 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:38.912506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:38.913332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:38.913648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:38.913921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:38.914616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:38.914920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:38.916336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:38.933104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 634/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:38.939347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:39.061212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4ab76-1212-4ca4-aef5-85650e3a5e89 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:39.093717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:39.102147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:39.102782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:39.103051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:39.103354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:39.111877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:39.113573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:21:39.126139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:39.127300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:39.128299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 619/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:39.137571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:39.201057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:39.256449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca5513-38ea-4d45-a026-268deab71e21 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:39.307128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:39.307128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:39.307128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:39.307128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:39.307585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:39.307585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:39.307585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:39.331135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 635/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 14:21:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:39.334734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:39.462716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0342f1-5b68-4e92-9b22-e67231d38ce3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:39.505287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:39.505984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:39.506261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:39.506648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:39.576698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:39.589753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:39.591171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:39.703520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 620/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:21:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:21:39.710907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:39.823027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34877a6a-343c-47de-9088-b08f0e452d3d None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:39.837254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:39.862048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:39.862670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:39.862948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:39.863940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:39.875370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:39.876808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:39.895666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:39.896694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:39.898074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:39.898074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:39.898074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:39.898074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 636/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:39.902874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:40.015163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:40.031556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619fbde-a0f8-485f-8910-f514044a20c9 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:40.071740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:40.072406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:40.072611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:40.072966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:40.073736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:40.074115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:40.075525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:40.088141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 621/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:40.093841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:40.202906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0868-2a69-44ac-b405-b7edf9245b35 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:40.239744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:40.240378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:40.240660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:40.240935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:40.247835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:40.249277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:21:40.260449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:40.261582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:40.262607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 637/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:40.268332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:40.381417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024bceec-f133-45c7-9f1a-a3bfcb46b9dd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:40.422615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:40.423588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:40.424065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:40.424346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:40.425267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:40.425707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:40.427613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0c540-d120-4f4f-ae99-a278c4d0f00a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:40.443328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 622/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:40.450015 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:40.483191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:40.580976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e37b5-0bb0-4754-8cc1-59fe560d88c0 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:40.615975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:40.616619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:40.616958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:40.617229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:40.617957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=90fd9b3148004cc2b8d14bced329b10b) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:40.618303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:40.619834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9bd23-aa71-44aa-bb8b-63ae26921690 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:40.625522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 638/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:21:40 2022] GET /identity/v3/role_assignments?user.id=90fd9b3148004cc2b8d14bced329b10b&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:42.683659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d67f6-2358-4aec-a730-4506270a707e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:42.684230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d67f6-2358-4aec-a730-4506270a707e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:42.684230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d67f6-2358-4aec-a730-4506270a707e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:42.685530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 623/1261] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:42.690640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:42.691024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:42.691214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:42.709414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:42.710594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:42.775070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 639/1262] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:42.797544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d67f6-2358-4aec-a730-4506270a707e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:42.809519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:42.900220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d67f6-2358-4aec-a730-4506270a707e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:42.933900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:42.934458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:42.934740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:42.935015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:42.944956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:21:42.949601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:42.950680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:42.956166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 624/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:21:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:42.956876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:42.975622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:43.054553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e1052-058e-4fdb-9b16-69e06f5c2b0a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:43.087635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:43.088229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:43.088592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:43.088772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:43.089423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:43.089722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:43.091076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:43.099429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 640/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:43.105431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:43.170868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:43.204148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0020-8785-4b95-8e32-e1d363eade1f None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:43.237329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:43.237948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:43.238252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:43.238530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:43.249051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:43.250508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:43.264096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:43.265176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:43.266081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:43.266081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:43.266081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:43.266081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 625/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:43.271652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:43.350006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:43.368767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdb255-e448-41d8-8d7d-1d40d7520ea9 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:43.401272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:43.401863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:43.402136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:43.402403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:43.403034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:43.403360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:43.404711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:43.416018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 641/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:43.424322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:43.544722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513d387-8e59-43dc-9693-a18dde6892cd None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:43.564364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:43.578756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:43.579412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:43.579693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:43.579974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:43.587787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:43.589226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:21:43.599932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:43.601011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:43.601934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 626/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:43.606871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:43.704485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a28b11-786e-48ca-b04e-6b375eaaee1b None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:43.737473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:43.738575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:43.738852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:43.739145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:43.739864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:43.740160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:43.741570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:43.749986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:43.759765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 642/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 14:21:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:43.765389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:43.864331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39c1e1-4d74-4a3d-830c-db0306912e69 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:43.897450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:43.898073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:43.898352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:43.898625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:43.908857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:43.910293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:43.923821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:43.924878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:43.926598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:43.926598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:43.926598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:43.926598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 627/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 14:21:43.931391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:44.030136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf6fd2-80a5-4cd2-9edf-510528092af1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:44.062894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:44.063520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:44.063800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:44.064073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:44.064709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:44.065011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:44.066986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:44.078505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 643/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:44.085011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:44.122947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:44.191838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f31d0f-83ae-496b-a991-64b0111734e3 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:44.225078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:44.225689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:44.225965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:44.226235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:44.232733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:44.234217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:21:44.245949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:44.246892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:44.247884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 628/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:21:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:44.253402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:44.317308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:44.360460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1f004-c028-45cb-83a4-b6355e8b23c5 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:44.394258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:44.394784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:44.395059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:44.395359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:44.396036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:44.396338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:44.397756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397a1c7-5fdb-4e5c-8b0c-ff66da9b9a84 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:44.410568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 644/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:21:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:44.416574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:44.521249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:44.530930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726cbe79-6dd9-4b56-9a34-0ae441bb4941 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:44.565463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c8a8d-7e1a-46d9-98d8-1d2e04c340bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:44.566120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c8a8d-7e1a-46d9-98d8-1d2e04c340bf None admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:44.566405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c8a8d-7e1a-46d9-98d8-1d2e04c340bf None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:44.566700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c8a8d-7e1a-46d9-98d8-1d2e04c340bf None admin] PATH_INFO: `/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/90fd9b3148004cc2b8d14bced329b10b/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:44.568609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c8a8d-7e1a-46d9-98d8-1d2e04c340bf None admin] RBAC: Authorizing `identity:create_grant(project_id=ad4e032807bf4ffb9242bf1c4e011a9f, user_id=90fd9b3148004cc2b8d14bced329b10b, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:44.570041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c8a8d-7e1a-46d9-98d8-1d2e04c340bf None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:44.583900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 629/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Mon Aug 8 14:21:44 2022] PUT /identity/v3/projects/ad4e032807bf4ffb9242bf1c4e011a9f/users/90fd9b3148004cc2b8d14bced329b10b/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:21:46.559727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465025-fd6f-4ae6-844f-a899b52300ea None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:46.560259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465025-fd6f-4ae6-844f-a899b52300ea None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:46.560259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01465025-fd6f-4ae6-844f-a899b52300ea None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:46.561598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 645/1274] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:46.566906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:46.567347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:46.567496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:46.585253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:46.586447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:46.651749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 630/1275] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:46.674784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01465025-fd6f-4ae6-844f-a899b52300ea None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:46.775009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01465025-fd6f-4ae6-844f-a899b52300ea None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:46.792605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:46.810044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:46.810624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:46.810905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:46.811209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:46.821330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 14:21:46.825972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:46.827025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:46.827920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 646/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Mon Aug 8 14:21:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:46.833580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:46.913356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:46.931885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d997f89-5aac-472c-9b41-a66c556ee696 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:46.968514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:46.969111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:46.969388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:46.969657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:46.970311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:46.970612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:46.971998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:46.980749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 631/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Mon Aug 8 14:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:46.986628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:47.084069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16bf3d6-2415-4906-a634-054eda7e9511 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:47.116496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:47.117106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:47.117381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:47.117648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:47.127733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:47.129670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:47.143078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:47.144746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:47.145765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:47.145765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:47.145765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:47.145765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 647/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:47.150463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:47.246501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce444e5c-5b59-4d96-8c4c-08d722876ae8 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:47.279969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:47.280557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:47.280836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:47.281106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:47.281730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:47.282029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:47.283410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:47.294665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 632/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:47.300427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:47.330928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:47.399755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b529d-1132-40bf-a0ba-4f914a41af99 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:47.431914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:47.432525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:47.432803 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:47.433071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] PATH_INFO: `/v3/users/placement` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:47.440292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:47.442197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 14:21:47.452836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:47.453956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:47.455148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 648/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:47.460769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:47.466167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:47.574049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142060f-8c45-41fc-abed-08e407af26fb None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:47.606284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:47.606881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:47.607184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:47.607458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:47.608171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:47.608474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:47.609910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:47.626641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 633/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Mon Aug 8 14:21:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:47.633092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:47.723413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:47.733020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f910210f-e49b-4fa8-9070-3de65e2b01ff None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:47.765988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:47.766584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:47.766864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:47.767160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:47.832919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:47.834371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:47.923671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:47.935586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 649/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Mon Aug 8 14:21:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 14:21:47.941609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:48.002933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:48.040417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53027-eecb-4da1-9149-cf8ec35baaa4 None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:48.073291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:48.073899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:48.074174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:48.074441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:48.084330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:48.086046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 14:21:48.100625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:48.101747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 14:21:48.102656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 14:21:48.102656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 14:21:48.102656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:48.102656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 634/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:48.107764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:48.202955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56888a-4b48-4ab8-b2e1-836a0400087c None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:48.236134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:48.236731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:48.237010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:48.237281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:48.238520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:48.238824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:48.240214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:48.251228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 650/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Mon Aug 8 14:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:21:48.257004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:48.355646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e69e83-adee-42c0-b4af-c4fd2a03e60b None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:48.368926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:48.388400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:48.389020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:48.389302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:48.389575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:48.396490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:48.397928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 14:21:48.408706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:48.410315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: ERROR keystone.server.flask.application Aug 08 14:21:48.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 635/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Mon Aug 8 14:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:48.416640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:48.467557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:48.523732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbefee-4285-4a07-a6ec-22ad53c158f2 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:48.557813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:48.558423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:48.558704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:48.558978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:48.559700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:48.560012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:48.561455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a9a87-9c16-42ae-8af5-b20836ec772a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:48.573328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 651/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Mon Aug 8 14:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:48.579388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:48.675878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bbbb9-aab9-4a4e-9898-19a7531f915d None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:48.708803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:48.709388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:48.709663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:48.709930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:48.710643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] RBAC: Adding query filter params (role.id=2708400faa2c49aaa3f8bf315b82dd8b, scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f, user.id=90fd9b3148004cc2b8d14bced329b10b) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:48.710948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:48.712394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180340ab-8ac7-4a33-94d9-59727826c8e7 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:48.718017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 636/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Mon Aug 8 14:21:48 2022] GET /identity/v3/role_assignments?user.id=90fd9b3148004cc2b8d14bced329b10b&scope.project.id=ad4e032807bf4ffb9242bf1c4e011a9f&role.id=2708400faa2c49aaa3f8bf315b82dd8b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:50.712910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309791-b28e-417f-877b-5e66fff8fdb8 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:50.713497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309791-b28e-417f-877b-5e66fff8fdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:50.713497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23309791-b28e-417f-877b-5e66fff8fdb8 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:50.714779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 652/1288] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:50.719744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:50.720167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:50.720291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:50.738954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:50.740199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:50.769444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:50.812727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 637/1289] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:50.835516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23309791-b28e-417f-877b-5e66fff8fdb8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:50.935740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23309791-b28e-417f-877b-5e66fff8fdb8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:50.946571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:50.969738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:50.970287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:50.970562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:50.970836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:50.971441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:50.972792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:50.981198 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:50.982472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:50.983660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 653/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:21:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:50.988249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:51.083874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ce0a7-4255-4b3c-82fe-987d7493c63d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:51.117129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:51.117720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:51.117995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:51.118264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:51.118880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:51.119212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:51.120642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec2023-bf1f-44b7-9ffb-eae6c0563c12 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:51.125280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 638/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 14:21:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:51.131576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:51.228386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cd59d-aa5e-4ab2-a6d7-ea22ad3161a1 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:51.261434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:51.262040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:51.262336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:51.262727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] PATH_INFO: `/v3/services` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:51.263400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:51.263699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:51.265060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efc37c9-48f4-432a-9613-273b06267ac6 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:51.269428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 654/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 14:21:51 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:53.267408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8c26b-02f5-412f-abed-71f889d0106c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:53.267930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8c26b-02f5-412f-abed-71f889d0106c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:53.267930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8c26b-02f5-412f-abed-71f889d0106c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:53.269179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 639/1293] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:53.274348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25341a4c-5e16-430c-abd8-504a6db217fc None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:53.274768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25341a4c-5e16-430c-abd8-504a6db217fc None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:53.274888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25341a4c-5e16-430c-abd8-504a6db217fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:53.292984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-25341a4c-5e16-430c-abd8-504a6db217fc None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:53.294149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-25341a4c-5e16-430c-abd8-504a6db217fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:53.323244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:53.360045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 655/1294] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:53.382759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8c26b-02f5-412f-abed-71f889d0106c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:53.413338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:53.486986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8c26b-02f5-412f-abed-71f889d0106c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:53.528768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321771c3-45e9-4f3c-a72a-4ba013704a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:53.529562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321771c3-45e9-4f3c-a72a-4ba013704a83 None admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:53.529964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321771c3-45e9-4f3c-a72a-4ba013704a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:53.530357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321771c3-45e9-4f3c-a72a-4ba013704a83 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:53.531131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321771c3-45e9-4f3c-a72a-4ba013704a83 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:53.532714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321771c3-45e9-4f3c-a72a-4ba013704a83 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:53.548130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 640/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Mon Aug 8 14:21:53 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:21:55.529292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac7423-5d12-4fe1-a585-48729e9ebef4 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:55.529849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac7423-5d12-4fe1-a585-48729e9ebef4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:55.529941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac7423-5d12-4fe1-a585-48729e9ebef4 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:55.531254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 656/1296] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:55.536426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516933da-f31b-4840-809e-04f470f98a4b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:55.536847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516933da-f31b-4840-809e-04f470f98a4b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:55.536971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516933da-f31b-4840-809e-04f470f98a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:55.555192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-516933da-f31b-4840-809e-04f470f98a4b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:55.556368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-516933da-f31b-4840-809e-04f470f98a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:55.626200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 641/1297] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:55 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:55.649676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac7423-5d12-4fe1-a585-48729e9ebef4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:55.750287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac7423-5d12-4fe1-a585-48729e9ebef4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:55.783896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:55.784615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:55.784895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:55.785163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:55.785754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:55.787339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:55.796376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:55.798078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:55.799685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 657/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:21:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:55.804529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516933da-f31b-4840-809e-04f470f98a4b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:55.823006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:55.904150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516933da-f31b-4840-809e-04f470f98a4b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:55.937355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:55.937951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:55.938235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:55.938515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:55.939164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:55.939474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:55.941464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:55.946917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 642/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 14:21:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:55.952875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:55.983474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:56.052193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bdca4-198f-4dbf-9b65-d5e1f438de1a None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:56.084673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:56.085270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:56.085549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:56.085818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:56.086512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] RBAC: Adding query filter params (interface=public, service_id=02d8c9ef740b4224b0de44452192d3bf, region_id=RegionOne) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:56.086818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:56.088210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed27799-9408-4cfb-aafc-3eeabd8b889c None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:56.093091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 658/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Mon Aug 8 14:21:55 2022] GET /identity/v3/endpoints?service_id=02d8c9ef740b4224b0de44452192d3bf&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:56.099219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:56.178773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:56.196764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94cfa14-6464-4a9c-865b-e8fd84b7578a None admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:56.229356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37d228-76ca-4313-9466-c39088be2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:56.230007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37d228-76ca-4313-9466-c39088be2185 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:56.230292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37d228-76ca-4313-9466-c39088be2185 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:56.230563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37d228-76ca-4313-9466-c39088be2185 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:56.231223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37d228-76ca-4313-9466-c39088be2185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:56.232567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37d228-76ca-4313-9466-c39088be2185 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:56.239788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 643/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Mon Aug 8 14:21:56 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:21:58.214657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92991861-cc09-4c80-bbf2-fd8272d5f5dc None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:58.215195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92991861-cc09-4c80-bbf2-fd8272d5f5dc None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:58.215195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92991861-cc09-4c80-bbf2-fd8272d5f5dc None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:58.216483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 659/1302] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:21:58.222282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:58.222733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:58.222833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:58.240335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:21:58.241513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:21:58.306336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 644/1303] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:21:58 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 14:21:58.328638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92991861-cc09-4c80-bbf2-fd8272d5f5dc None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:58.423620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:58.431358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92991861-cc09-4c80-bbf2-fd8272d5f5dc None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:58.466494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:58.467147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:58.467443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:58.467752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:58.468398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:58.469938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 14:21:58.479562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 14:21:58.480926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application with Lock( Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self._enter() Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application return self.creator() Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application created_value = creator( Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: ERROR keystone.server.flask.application Aug 08 14:21:58.481776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 660/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Mon Aug 8 14:21:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 14:21:58.487140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:58.572521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:21:58.595480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092c0d1-ec8f-4f87-8cc9-4d34b8938c0e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:58.629053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:58.629686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:58.629967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:58.630244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] PATH_INFO: `/v3/services` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:58.630874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:21:58.631217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:58.632615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1788a00-e720-4641-b1b9-b234b9167a66 None admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:58.637622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 645/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Mon Aug 8 14:21:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:21:58.643423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:21:58.741063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c79c3-e1c4-4fdd-8648-c4cd71df9099 None admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:21:58.774196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d921e-0da8-49c9-9314-39c1636eaf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:21:58.774787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d921e-0da8-49c9-9314-39c1636eaf64 None admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:21:58.775065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d921e-0da8-49c9-9314-39c1636eaf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:21:58.775366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d921e-0da8-49c9-9314-39c1636eaf64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:21:58.776023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d921e-0da8-49c9-9314-39c1636eaf64 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:21:58.777385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d921e-0da8-49c9-9314-39c1636eaf64 None admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:21:58.797963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 661/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:21:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:22:08.634692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=30 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:22:08.793752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:02.496294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e62cd-fbe7-4b8e-b349-91f7732b189c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:02.496720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e62cd-fbe7-4b8e-b349-91f7732b189c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:02.496946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e62cd-fbe7-4b8e-b349-91f7732b189c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:02.498317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 646/1307] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:02.503976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94b24b-0d76-43e9-8321-a3e1d05298be None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:02.504357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94b24b-0d76-43e9-8321-a3e1d05298be None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:02.504515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94b24b-0d76-43e9-8321-a3e1d05298be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:02.602504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-be94b24b-0d76-43e9-8321-a3e1d05298be None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:02.603771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-be94b24b-0d76-43e9-8321-a3e1d05298be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:02.743486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 662/1308] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:02.794840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d9b4d-b3ae-4631-bf95-c923cf41fbfc None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:02.795152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d9b4d-b3ae-4631-bf95-c923cf41fbfc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:02.795418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d9b4d-b3ae-4631-bf95-c923cf41fbfc None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:02.797478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 647/1309] 10.222.0.50 () {54 vars in 965 bytes} [Mon Aug 8 14:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:02.804032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de7536-4843-4a68-b8f3-9cbaa26b1920 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:02.804247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de7536-4843-4a68-b8f3-9cbaa26b1920 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:02.804686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de7536-4843-4a68-b8f3-9cbaa26b1920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:02.896483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-21de7536-4843-4a68-b8f3-9cbaa26b1920 None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:02.897702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-21de7536-4843-4a68-b8f3-9cbaa26b1920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:02.920506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:02.986420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 663/1310] 10.222.0.50 () {58 vars in 1062 bytes} [Mon Aug 8 14:23:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:02.995408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d9b4d-b3ae-4631-bf95-c923cf41fbfc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:02.997524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7d2d9b4d-b3ae-4631-bf95-c923cf41fbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:03.121977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d9b4d-b3ae-4631-bf95-c923cf41fbfc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:03.207455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:03.208069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:03.208349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:03.208751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:03.210367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:03.275552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:03.345297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:03.346970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3066df-69b7-4c97-b58d-4db47eccfe78 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:03.370029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 648/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:05.673616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e460-ef27-4b7c-8e82-fa19e14864eb None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:05.674403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e460-ef27-4b7c-8e82-fa19e14864eb None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:05.674496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e460-ef27-4b7c-8e82-fa19e14864eb None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:05.675826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 664/1312] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:05.681209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e9b2-ce4d-4078-bb62-4f01c8307efe None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:05.681661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e9b2-ce4d-4078-bb62-4f01c8307efe None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:05.681752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e9b2-ce4d-4078-bb62-4f01c8307efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:05.766239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1e44e9b2-ce4d-4078-bb62-4f01c8307efe None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:05.767485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1e44e9b2-ce4d-4078-bb62-4f01c8307efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:05.879314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 649/1313] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:05.934197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf761c-d65d-4157-bbca-881d71e7ded1 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:05.934403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf761c-d65d-4157-bbca-881d71e7ded1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:05.934657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf761c-d65d-4157-bbca-881d71e7ded1 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:05.936060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 665/1314] 10.222.0.50 () {54 vars in 965 bytes} [Mon Aug 8 14:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:05.942853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c64cb0-e68e-4bf2-ac87-58fae29b91c7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:05.943167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c64cb0-e68e-4bf2-ac87-58fae29b91c7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:05.943446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c64cb0-e68e-4bf2-ac87-58fae29b91c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:05.955585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:06.022574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-50c64cb0-e68e-4bf2-ac87-58fae29b91c7 None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:06.023813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-50c64cb0-e68e-4bf2-ac87-58fae29b91c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:06.101297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 650/1315] 10.222.0.50 () {58 vars in 1062 bytes} [Mon Aug 8 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:06.111254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf761c-d65d-4157-bbca-881d71e7ded1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:06.112819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-10cf761c-d65d-4157-bbca-881d71e7ded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:06.219169 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf761c-d65d-4157-bbca-881d71e7ded1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:06.293757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:06.296521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:06.297313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:06.297793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:06.298255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:06.300192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:06.408811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:06.410248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31fc98d-ee61-4198-9447-e5d1b518e014 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:06.428754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 666/1316] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:08.948314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761dbaf-3015-457f-910d-f7b5d04a0c8c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:08.948873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761dbaf-3015-457f-910d-f7b5d04a0c8c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:08.948873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761dbaf-3015-457f-910d-f7b5d04a0c8c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:08.950185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 651/1317] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:08.955343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e8794-5404-4917-8cae-03b6e28e3fbc None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:08.955734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e8794-5404-4917-8cae-03b6e28e3fbc None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:08.955898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e8794-5404-4917-8cae-03b6e28e3fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:09.020139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bf9e8794-5404-4917-8cae-03b6e28e3fbc None None] MFA Rules not processed for user `6278e07627da40afa46d323a5ce985e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:09.021346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bf9e8794-5404-4917-8cae-03b6e28e3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:09.073620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:09.118366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 667/1318] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:09.158476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761dbaf-3015-457f-910d-f7b5d04a0c8c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:09.160055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0761dbaf-3015-457f-910d-f7b5d04a0c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:09.265464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761dbaf-3015-457f-910d-f7b5d04a0c8c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:09.268550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:09.339758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:09.340446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:09.340636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:09.340908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:09.342508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:09.453105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:09.454437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8011d6-3d78-4774-aa07-a4129a3490cc service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:09.472999 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 652/1319] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:12.035414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf365e-7411-4116-97e8-220e6bd9dc82 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:12.035965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf365e-7411-4116-97e8-220e6bd9dc82 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:12.035965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf365e-7411-4116-97e8-220e6bd9dc82 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:12.037288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 668/1320] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:12.042536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f95ec-10c6-40fb-b726-6f2306c1e830 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:12.042995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f95ec-10c6-40fb-b726-6f2306c1e830 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:12.043129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f95ec-10c6-40fb-b726-6f2306c1e830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:12.083920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:12.109276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-461f95ec-10c6-40fb-b726-6f2306c1e830 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:12.110478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-461f95ec-10c6-40fb-b726-6f2306c1e830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:12.204424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 653/1321] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:12.243256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf365e-7411-4116-97e8-220e6bd9dc82 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:12.244787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b7bf365e-7411-4116-97e8-220e6bd9dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:12.346355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf365e-7411-4116-97e8-220e6bd9dc82 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:12.417053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:12.417648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:12.417926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:12.418193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:12.419805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:12.427317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:12.527547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:12.528898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee81aa-fb18-4c2a-833c-d5527d3d66a9 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:12.547060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 669/1322] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:15.260040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e1b4-b068-4ccb-99a6-b3eb8195e4f6 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:15.260544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e1b4-b068-4ccb-99a6-b3eb8195e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:15.260544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e1b4-b068-4ccb-99a6-b3eb8195e4f6 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:15.262000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 654/1323] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:15.267687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2093ead-d7d3-4d1d-af66-c1773ba7a45a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:15.268137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2093ead-d7d3-4d1d-af66-c1773ba7a45a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:15.268279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2093ead-d7d3-4d1d-af66-c1773ba7a45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:15.337684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e2093ead-d7d3-4d1d-af66-c1773ba7a45a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:15.338917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e2093ead-d7d3-4d1d-af66-c1773ba7a45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:15.404996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:15.448305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 670/1324] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:15.486238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e1b4-b068-4ccb-99a6-b3eb8195e4f6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:15.487872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f717e1b4-b068-4ccb-99a6-b3eb8195e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:15.597118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e1b4-b068-4ccb-99a6-b3eb8195e4f6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:15.610192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:15.674067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:15.674679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:15.674962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:15.675271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:15.676870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:15.785803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:15.787209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb92c-4094-49ba-baed-b7755efdf631 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:15.811597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 655/1325] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:17.886261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc974d9-bb7c-492d-91d2-01a46aa7670d None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:17.886734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc974d9-bb7c-492d-91d2-01a46aa7670d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:17.886824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc974d9-bb7c-492d-91d2-01a46aa7670d None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:17.888280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 671/1326] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:17.893813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e25a99-edf2-4b7c-8bd0-df2e4f75079c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:17.894239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e25a99-edf2-4b7c-8bd0-df2e4f75079c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:17.894371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e25a99-edf2-4b7c-8bd0-df2e4f75079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:17.952733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:17.971779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-45e25a99-edf2-4b7c-8bd0-df2e4f75079c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:17.973003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-45e25a99-edf2-4b7c-8bd0-df2e4f75079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:18.072082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 656/1327] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:18.108827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc974d9-bb7c-492d-91d2-01a46aa7670d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:18.110355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4dc974d9-bb7c-492d-91d2-01a46aa7670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:18.219721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc974d9-bb7c-492d-91d2-01a46aa7670d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:18.296209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:18.296812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:18.297093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:18.297372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:18.299271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:18.316772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:18.421524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:18.422975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327d22c-5d6b-471a-a91a-6411db16a67f service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:18.441542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 672/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:20.494989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05211ebb-f14f-45d4-9120-98e00a6418d5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:20.495534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05211ebb-f14f-45d4-9120-98e00a6418d5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:20.495534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05211ebb-f14f-45d4-9120-98e00a6418d5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:20.496921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 657/1329] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:20.502125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f911fa-a910-47bc-8e66-8f5c16cd1ae4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:20.502583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f911fa-a910-47bc-8e66-8f5c16cd1ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:20.502706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f911fa-a910-47bc-8e66-8f5c16cd1ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:20.571773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f8f911fa-a910-47bc-8e66-8f5c16cd1ae4 None None] MFA Rules not processed for user `6278e07627da40afa46d323a5ce985e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:20.572968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f8f911fa-a910-47bc-8e66-8f5c16cd1ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:20.647871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:20.682605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 673/1330] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:20.723507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05211ebb-f14f-45d4-9120-98e00a6418d5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:20.725066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-05211ebb-f14f-45d4-9120-98e00a6418d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:20.831216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05211ebb-f14f-45d4-9120-98e00a6418d5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:20.861991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:20.921711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:20.922315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:20.922667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:20.922952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:20.924839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:21.062493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:21.064095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886f544-6ae2-4efc-a524-0e2ef0a6be6b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:21.083995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 658/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:23.322117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f8b1-c3c3-4ecf-97b9-cded46e10794 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:23.322694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f8b1-c3c3-4ecf-97b9-cded46e10794 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:23.322694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410f8b1-c3c3-4ecf-97b9-cded46e10794 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:23.324023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 674/1332] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:23.329218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f68f6f-64d7-4f57-a25d-09909929690c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:23.329692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f68f6f-64d7-4f57-a25d-09909929690c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:23.329865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f68f6f-64d7-4f57-a25d-09909929690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:23.395297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:23.397950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-31f68f6f-64d7-4f57-a25d-09909929690c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:23.399740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-31f68f6f-64d7-4f57-a25d-09909929690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:23.496377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 659/1333] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:23.533155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f8b1-c3c3-4ecf-97b9-cded46e10794 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:23.534697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4410f8b1-c3c3-4ecf-97b9-cded46e10794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:23.636831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410f8b1-c3c3-4ecf-97b9-cded46e10794 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:23.708734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:23.709338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:23.709613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:23.709883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:23.711573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:23.738211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:23.822728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:23.824186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe66e2-b831-479d-81c9-8335cead4c61 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:23.843007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 675/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:25.872771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de893ff9-2af7-4706-b135-61af9a4162f2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:25.873447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de893ff9-2af7-4706-b135-61af9a4162f2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:25.873447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de893ff9-2af7-4706-b135-61af9a4162f2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:25.875320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 660/1335] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:25.880809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce1fc-4c92-4a5c-b0c4-af89eac8a534 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:25.881309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce1fc-4c92-4a5c-b0c4-af89eac8a534 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:25.881421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce1fc-4c92-4a5c-b0c4-af89eac8a534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:25.946085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ef5ce1fc-4c92-4a5c-b0c4-af89eac8a534 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:25.947302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ef5ce1fc-4c92-4a5c-b0c4-af89eac8a534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:26.020164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:26.042153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 676/1336] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:26.082746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de893ff9-2af7-4706-b135-61af9a4162f2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:26.084349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-de893ff9-2af7-4706-b135-61af9a4162f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:26.189107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de893ff9-2af7-4706-b135-61af9a4162f2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:26.216999 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:26.262664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:26.263299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:26.263585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:26.263857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:26.265495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:26.373226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:26.374699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7511eb9c-24a4-466f-abc3-25a563790479 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:26.392436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 661/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:28.753256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffbe1b-e408-496b-8314-f27e3be1a15a None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:28.753821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffbe1b-e408-496b-8314-f27e3be1a15a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:28.753821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffbe1b-e408-496b-8314-f27e3be1a15a None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:28.755161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 677/1338] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:28.760300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15699ca-b3c7-4885-a19b-bd41660bf7fc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:28.760748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15699ca-b3c7-4885-a19b-bd41660bf7fc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:28.760864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15699ca-b3c7-4885-a19b-bd41660bf7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:28.827350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b15699ca-b3c7-4885-a19b-bd41660bf7fc None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:28.828602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b15699ca-b3c7-4885-a19b-bd41660bf7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:28.837112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:28.925377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 662/1339] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:28.961336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffbe1b-e408-496b-8314-f27e3be1a15a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:28.962858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9bffbe1b-e408-496b-8314-f27e3be1a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:29.065384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffbe1b-e408-496b-8314-f27e3be1a15a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:29.136562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:29.137153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:29.137425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:29.137690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:29.139919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:29.174497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:29.247246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:29.249358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c57c3-5fc6-49ca-a741-6a2e7a4ef704 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:29.269006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 678/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:23:33.592594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1fc3-5087-4228-9ff0-5367387b87e7 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:33.593292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1fc3-5087-4228-9ff0-5367387b87e7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:33.593292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1fc3-5087-4228-9ff0-5367387b87e7 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:33.594498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 663/1341] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:33.599829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e096c3-4aa3-49f0-849b-cc2fece67150 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:33.600288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e096c3-4aa3-49f0-849b-cc2fece67150 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:33.600405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e096c3-4aa3-49f0-849b-cc2fece67150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:33.666201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-94e096c3-4aa3-49f0-849b-cc2fece67150 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:33.667426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-94e096c3-4aa3-49f0-849b-cc2fece67150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:33.756775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:33.768598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 679/1342] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:33.809194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1fc3-5087-4228-9ff0-5367387b87e7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:33.810814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7d2b1fc3-5087-4228-9ff0-5367387b87e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:33.921424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1fc3-5087-4228-9ff0-5367387b87e7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:33.961063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:33.998586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:33.999517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:33.999804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:34.000080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:34.001693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:34.111048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:34.112502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe65096-5c59-445c-a760-f054b1fcad22 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:34.129928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 664/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:36.800211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657eba0-8611-4063-a73b-878dac698b89 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:36.800823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657eba0-8611-4063-a73b-878dac698b89 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:36.800823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657eba0-8611-4063-a73b-878dac698b89 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:36.802161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 680/1344] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:36.807915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ffdcd-1d3b-450d-aa16-3215206bffe2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:36.808468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ffdcd-1d3b-450d-aa16-3215206bffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:36.808468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ffdcd-1d3b-450d-aa16-3215206bffe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:36.880073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-414ffdcd-1d3b-450d-aa16-3215206bffe2 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:36.881281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-414ffdcd-1d3b-450d-aa16-3215206bffe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:36.900885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:36.986200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 665/1345] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:37.023948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657eba0-8611-4063-a73b-878dac698b89 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:37.025487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2657eba0-8611-4063-a73b-878dac698b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:37.130907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657eba0-8611-4063-a73b-878dac698b89 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:37.205996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:37.206596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:37.206872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:37.207173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:37.208787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:37.255769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:37.327384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:37.328886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4526b9a-ccd5-4e1f-a409-10051e3372d0 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:37.349444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 681/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:41.536961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce34c-bc82-4cde-89c3-fa3681485a50 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:41.537524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce34c-bc82-4cde-89c3-fa3681485a50 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:41.537524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cce34c-bc82-4cde-89c3-fa3681485a50 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:41.538840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 666/1347] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:41.543970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846efef-7702-42b4-97f8-71081418db29 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:41.544366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846efef-7702-42b4-97f8-71081418db29 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:41.545049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846efef-7702-42b4-97f8-71081418db29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:41.611952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1846efef-7702-42b4-97f8-71081418db29 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:41.613163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1846efef-7702-42b4-97f8-71081418db29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:41.711780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:41.714275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 682/1348] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:41.753943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cce34c-bc82-4cde-89c3-fa3681485a50 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:41.754959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-48cce34c-bc82-4cde-89c3-fa3681485a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:41.881160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cce34c-bc82-4cde-89c3-fa3681485a50 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:41.933535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:41.961664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:41.962346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:41.962571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:41.962836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:41.964566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:42.082033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:42.083529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885404a-74e5-4c97-8cc9-1655eeac60ad service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:42.102767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 667/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:23:46.987874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa96a7-1dc2-4fda-ac97-9fbbec111a1e None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:46.988349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa96a7-1dc2-4fda-ac97-9fbbec111a1e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:46.988428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baa96a7-1dc2-4fda-ac97-9fbbec111a1e None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:46.990077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 683/1350] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:46.995459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a54f6-8ad6-40f1-873a-c788da977f9c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:46.995873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a54f6-8ad6-40f1-873a-c788da977f9c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:46.996058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a54f6-8ad6-40f1-873a-c788da977f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:47.077319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-026a54f6-8ad6-40f1-873a-c788da977f9c None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:47.078519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-026a54f6-8ad6-40f1-873a-c788da977f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:47.122017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:47.246663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 668/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:23:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:51.561397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4b4e4-6929-48c9-9479-ac1038be76ea None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:51.562058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4b4e4-6929-48c9-9479-ac1038be76ea None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:51.562058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4b4e4-6929-48c9-9479-ac1038be76ea None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:51.563483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 684/1352] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:23:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:51.569941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405d0d4-dcea-41f7-98f3-4b0ac14b57a6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:51.570435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405d0d4-dcea-41f7-98f3-4b0ac14b57a6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:51.570514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405d0d4-dcea-41f7-98f3-4b0ac14b57a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:51.661065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b405d0d4-dcea-41f7-98f3-4b0ac14b57a6 None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:51.661290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b405d0d4-dcea-41f7-98f3-4b0ac14b57a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:51.751848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:51.799009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 669/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:23:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:23:53.038862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc836f8d-347c-4ca1-8c77-92b3e21e37bf None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:53.039464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc836f8d-347c-4ca1-8c77-92b3e21e37bf None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:53.039464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc836f8d-347c-4ca1-8c77-92b3e21e37bf None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:53.040780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 685/1354] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:53.046216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e471f-9ddf-4cc5-9003-d6bd16d595a3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:53.046677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e471f-9ddf-4cc5-9003-d6bd16d595a3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:53.046775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e471f-9ddf-4cc5-9003-d6bd16d595a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:53.126106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-360e471f-9ddf-4cc5-9003-d6bd16d595a3 None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:53.127345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-360e471f-9ddf-4cc5-9003-d6bd16d595a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:53.257401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 670/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:56.519446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f68e7e-dc5d-494e-9e7c-18db18c3b081 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:56.520268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f68e7e-dc5d-494e-9e7c-18db18c3b081 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:56.520721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f68e7e-dc5d-494e-9e7c-18db18c3b081 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:56.522620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 686/1356] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:56.528344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342275e-f545-499d-a4b4-97d175d9597f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:56.529189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342275e-f545-499d-a4b4-97d175d9597f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:56.529469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4342275e-f545-499d-a4b4-97d175d9597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:56.568071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:56.604364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4342275e-f545-499d-a4b4-97d175d9597f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:56.605592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4342275e-f545-499d-a4b4-97d175d9597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:56.713066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 671/1357] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:56.767508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b0be1-6d1a-4ad9-bbbf-3afe0d04c752 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:56.768032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b0be1-6d1a-4ad9-bbbf-3afe0d04c752 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:56.768686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b0be1-6d1a-4ad9-bbbf-3afe0d04c752 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:56.772170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 687/1358] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 14:23:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:56.778723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f23c1-815a-4228-8706-6b26db1b397b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:56.780066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f23c1-815a-4228-8706-6b26db1b397b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:56.782554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f23c1-815a-4228-8706-6b26db1b397b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:56.877307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e59f23c1-815a-4228-8706-6b26db1b397b None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:56.877307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e59f23c1-815a-4228-8706-6b26db1b397b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:56.934169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:57.009331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 672/1359] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:23:57.017306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b0be1-6d1a-4ad9-bbbf-3afe0d04c752 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:57.018899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9f8b0be1-6d1a-4ad9-bbbf-3afe0d04c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:57.171413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b0be1-6d1a-4ad9-bbbf-3afe0d04c752 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:23:57.250129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:57.295224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:23:57.295928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:57.296279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:57.296624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:57.298717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:57.406014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:23:57.407899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8121ba6-425c-4821-b983-1cbd5e8cc5d3 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:23:57.427397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 688/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:23:59.671442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68158f11-5084-4bbe-8d7c-fa0a5c35ffd3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:59.671981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68158f11-5084-4bbe-8d7c-fa0a5c35ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:59.672605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68158f11-5084-4bbe-8d7c-fa0a5c35ffd3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:59.674162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 673/1361] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:23:59.679717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd6b4-147e-48ac-836c-d950af2a9e93 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:23:59.680148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd6b4-147e-48ac-836c-d950af2a9e93 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:23:59.680243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdd6b4-147e-48ac-836c-d950af2a9e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:23:59.749804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d6bdd6b4-147e-48ac-836c-d950af2a9e93 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:23:59.751071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d6bdd6b4-147e-48ac-836c-d950af2a9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:23:59.792370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:23:59.877808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 689/1362] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:23:59.917333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68158f11-5084-4bbe-8d7c-fa0a5c35ffd3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:23:59.920678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-68158f11-5084-4bbe-8d7c-fa0a5c35ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:00.119508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68158f11-5084-4bbe-8d7c-fa0a5c35ffd3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:00.133617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:00.254842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:00.255795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:00.256232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:00.257689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:00.259631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:00.356621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:00.358138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1176eec-590f-4772-a390-a57dfd11a66b service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:00.378141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 674/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:02.779149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd331b-dde2-4f1b-bd4a-17a1be435ca4 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:02.780305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd331b-dde2-4f1b-bd4a-17a1be435ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:02.780953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd331b-dde2-4f1b-bd4a-17a1be435ca4 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:02.783238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 690/1364] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:02.788829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e7d0a-4bed-47d5-a305-31920e705b9e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:02.789311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e7d0a-4bed-47d5-a305-31920e705b9e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:02.789423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e7d0a-4bed-47d5-a305-31920e705b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:02.808114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:02.866007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-156e7d0a-4bed-47d5-a305-31920e705b9e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:02.869878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-156e7d0a-4bed-47d5-a305-31920e705b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:02.978351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 675/1365] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:24:03.030627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ff65-c18b-44c5-a146-421426460455 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:03.030904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ff65-c18b-44c5-a146-421426460455 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:03.031367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ff65-c18b-44c5-a146-421426460455 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:03.032884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 691/1366] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 14:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:03.038736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae29e7a-444b-49b7-b81e-75278388f265 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:03.039179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae29e7a-444b-49b7-b81e-75278388f265 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:03.039572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae29e7a-444b-49b7-b81e-75278388f265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:03.124168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bae29e7a-444b-49b7-b81e-75278388f265 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:03.125378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bae29e7a-444b-49b7-b81e-75278388f265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:03.164348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:03.269451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 676/1367] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 14:24:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:03.277656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ff65-c18b-44c5-a146-421426460455 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:03.279239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d3c7ff65-c18b-44c5-a146-421426460455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:03.444621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ff65-c18b-44c5-a146-421426460455 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:03.467378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:03.585367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:03.585983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:03.586391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:03.586673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:03.588300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:03.694434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:03.696064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c89c2-124c-430f-88df-d0d76f01e635 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:03.717723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 692/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:04.614305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763b0b8-2c70-49c3-9f18-a0a9c3ee85b2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:04.614900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763b0b8-2c70-49c3-9f18-a0a9c3ee85b2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:04.614900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763b0b8-2c70-49c3-9f18-a0a9c3ee85b2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:04.616511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 677/1369] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:04.621946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32662d5-a6c7-43bd-85b8-72c58041809e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:04.622375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32662d5-a6c7-43bd-85b8-72c58041809e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:04.622568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32662d5-a6c7-43bd-85b8-72c58041809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:04.655666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:04.695178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f32662d5-a6c7-43bd-85b8-72c58041809e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:04.696404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f32662d5-a6c7-43bd-85b8-72c58041809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:04.807915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 693/1370] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:05.164135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244bc2c-6367-4be2-943b-b9920b7a78f9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:05.164442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244bc2c-6367-4be2-943b-b9920b7a78f9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:05.164686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244bc2c-6367-4be2-943b-b9920b7a78f9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:05.166193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 678/1371] 10.222.0.50 () {54 vars in 964 bytes} [Mon Aug 8 14:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:05.171706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00462476-8f0b-44ba-a7c5-30605c2e6620 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:05.171998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00462476-8f0b-44ba-a7c5-30605c2e6620 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:05.172535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00462476-8f0b-44ba-a7c5-30605c2e6620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:05.252122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-00462476-8f0b-44ba-a7c5-30605c2e6620 None None] MFA Rules not processed for user `d22e2d936518407fba2d0be379598ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:05.253323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-00462476-8f0b-44ba-a7c5-30605c2e6620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:05.305276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:05.337246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 694/1372] 10.222.0.50 () {58 vars in 1061 bytes} [Mon Aug 8 14:24:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:05.345119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244bc2c-6367-4be2-943b-b9920b7a78f9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:05.346631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f244bc2c-6367-4be2-943b-b9920b7a78f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:05.456139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244bc2c-6367-4be2-943b-b9920b7a78f9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:05.498497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:05.538249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:05.538851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:05.539150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:05.539420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:05.541039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:05.663403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:05.664855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad505cdc-12b2-4429-854b-b54760a708c0 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:05.687217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 679/1373] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:05.837406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405934-df38-447b-be60-d83ce251a6c9 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:05.837871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405934-df38-447b-be60-d83ce251a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:05.838074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405934-df38-447b-be60-d83ce251a6c9 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:05.839367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 695/1374] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:05.846622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4a46-7a7d-4824-b88b-ee34c9a452ce None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:05.846899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4a46-7a7d-4824-b88b-ee34c9a452ce None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:05.847122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4a46-7a7d-4824-b88b-ee34c9a452ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:05.919256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a0dd4a46-7a7d-4824-b88b-ee34c9a452ce None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:05.920615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a0dd4a46-7a7d-4824-b88b-ee34c9a452ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:05.942203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:06.035111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 680/1375] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:06.080312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405934-df38-447b-be60-d83ce251a6c9 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:06.081835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ea405934-df38-447b-be60-d83ce251a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:06.232403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405934-df38-447b-be60-d83ce251a6c9 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:06.283364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:06.346357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:06.346993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:06.347309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:06.347582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:06.349197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:06.446410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:06.448034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045e926-b22f-43fc-8f0e-dddaaecf3916 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:06.465882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 696/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:08.513732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10571e25-a84b-4ceb-aba9-0e01568aa9f8 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:08.514256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10571e25-a84b-4ceb-aba9-0e01568aa9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:08.514256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10571e25-a84b-4ceb-aba9-0e01568aa9f8 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:08.515610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 681/1377] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:08.520828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195677-d530-4bdf-a93f-cf9bfcae2c2f None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:08.521241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195677-d530-4bdf-a93f-cf9bfcae2c2f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:08.521579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db195677-d530-4bdf-a93f-cf9bfcae2c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:08.588041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-db195677-d530-4bdf-a93f-cf9bfcae2c2f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:08.589254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-db195677-d530-4bdf-a93f-cf9bfcae2c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:08.600625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:08.724218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 697/1378] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:08.768972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10571e25-a84b-4ceb-aba9-0e01568aa9f8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:08.770501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10571e25-a84b-4ceb-aba9-0e01568aa9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:08.921828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10571e25-a84b-4ceb-aba9-0e01568aa9f8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:08.925731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:09.042022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:09.042022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:09.042022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:09.042022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:09.044541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:09.164968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:09.167376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cb99a-4124-4026-89b9-021495dd9022 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:09.189786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 682/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:11.253854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f7f28-01db-444e-adb3-60d53b2c318b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:11.254411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f7f28-01db-444e-adb3-60d53b2c318b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:11.254411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f7f28-01db-444e-adb3-60d53b2c318b None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:11.255915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 698/1380] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:11.261084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fa6b-e4f6-4d6c-9fc3-660630c2b3a6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:11.261532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fa6b-e4f6-4d6c-9fc3-660630c2b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:11.261632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76fa6b-e4f6-4d6c-9fc3-660630c2b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:11.274527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:11.340358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9d76fa6b-e4f6-4d6c-9fc3-660630c2b3a6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:11.341535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9d76fa6b-e4f6-4d6c-9fc3-660630c2b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:11.446134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 683/1381] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:11.491574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f7f28-01db-444e-adb3-60d53b2c318b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:11.493064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-985f7f28-01db-444e-adb3-60d53b2c318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:11.639230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:11.650690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f7f28-01db-444e-adb3-60d53b2c318b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:11.771538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:11.772145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:11.772424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:11.772699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:11.774304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:11.877080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:11.879105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ba75e-9f49-4070-b8ac-5f6c07f909f5 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:11.890537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:11.902889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 699/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:14.111761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9980f1-83a3-462a-b8fd-fc50e8031d23 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:14.112292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9980f1-83a3-462a-b8fd-fc50e8031d23 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:14.112402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9980f1-83a3-462a-b8fd-fc50e8031d23 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:14.113776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 684/1383] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:14.119884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38217fa5-ef2f-4287-964a-52f58bc6ed8b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:14.120435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38217fa5-ef2f-4287-964a-52f58bc6ed8b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:14.120589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38217fa5-ef2f-4287-964a-52f58bc6ed8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:14.195843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-38217fa5-ef2f-4287-964a-52f58bc6ed8b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:14.197145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-38217fa5-ef2f-4287-964a-52f58bc6ed8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:14.317493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 700/1384] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:24:14.365246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9980f1-83a3-462a-b8fd-fc50e8031d23 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:14.366826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6e9980f1-83a3-462a-b8fd-fc50e8031d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:14.466126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:14.534436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9980f1-83a3-462a-b8fd-fc50e8031d23 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:14.698755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:14.700495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:14.700893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:14.701309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:14.703854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:14.776825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:14.828790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:14.831235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262a35c-eaac-46eb-9701-20737e3479f7 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:14.861048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 685/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:16.928035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e7c1d-7951-4b61-a91d-10414803a92b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:16.928570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e7c1d-7951-4b61-a91d-10414803a92b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:16.928683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e7c1d-7951-4b61-a91d-10414803a92b None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:16.929977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 701/1386] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:16.936795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd2829-f7a9-4124-879b-4bd09b4b0ceb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:16.937204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd2829-f7a9-4124-879b-4bd09b4b0ceb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:16.937339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd2829-f7a9-4124-879b-4bd09b4b0ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:17.018652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-72dd2829-f7a9-4124-879b-4bd09b4b0ceb None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:17.019919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-72dd2829-f7a9-4124-879b-4bd09b4b0ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:17.119326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 686/1387] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:24:17.177967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e7c1d-7951-4b61-a91d-10414803a92b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:17.180281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fe4e7c1d-7951-4b61-a91d-10414803a92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:17.284674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:17.370414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e7c1d-7951-4b61-a91d-10414803a92b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:17.510706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:17.512068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:17.512570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:17.512956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:17.515305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:17.559133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:17.636029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:17.638293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ed59d-34a2-4b12-a9f8-9fff00e5b870 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:17.660734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 702/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:19.886900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373e51-1ee1-47c7-9e0e-e870e9ceecc7 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:19.887408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373e51-1ee1-47c7-9e0e-e870e9ceecc7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:19.887522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90373e51-1ee1-47c7-9e0e-e870e9ceecc7 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:19.888893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 687/1389] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:19.894264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4dcd5-83fc-4243-97fb-da2435ac8ff2 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:19.894685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4dcd5-83fc-4243-97fb-da2435ac8ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:19.894856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4dcd5-83fc-4243-97fb-da2435ac8ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:19.961445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4fb4dcd5-83fc-4243-97fb-da2435ac8ff2 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:19.962648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4fb4dcd5-83fc-4243-97fb-da2435ac8ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:20.062531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 703/1390] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:20.106274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90373e51-1ee1-47c7-9e0e-e870e9ceecc7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:20.108031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-90373e51-1ee1-47c7-9e0e-e870e9ceecc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:20.139857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:20.255752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90373e51-1ee1-47c7-9e0e-e870e9ceecc7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:20.358473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:20.370679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:20.371327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:20.371606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:20.371879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:20.373468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:20.463011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:20.465041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c7a3e-e701-4644-87bf-53294ee5074a service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:20.483542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 688/1391] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:20.714875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4055cb-ba77-42e5-b274-4fe13f631b56 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:20.715136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4055cb-ba77-42e5-b274-4fe13f631b56 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:20.715424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4055cb-ba77-42e5-b274-4fe13f631b56 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:20.717335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 704/1392] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:20.723552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ba49-56d1-4eff-8069-9962e1b43d8c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:20.723911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ba49-56d1-4eff-8069-9962e1b43d8c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:20.724231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1121ba49-56d1-4eff-8069-9962e1b43d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:20.794224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1121ba49-56d1-4eff-8069-9962e1b43d8c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:20.795482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1121ba49-56d1-4eff-8069-9962e1b43d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:20.855972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:20.901514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 689/1393] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:24:20.957708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ae5f-fcde-4238-adfe-c6fd7cad6eb3 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:20.958185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ae5f-fcde-4238-adfe-c6fd7cad6eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:20.958275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714ae5f-fcde-4238-adfe-c6fd7cad6eb3 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:20.959625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 705/1394] 10.222.0.50 () {54 vars in 967 bytes} [Mon Aug 8 14:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:20.964612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7f527-b400-445a-83b2-6a8cdd8357fa None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:20.965129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7f527-b400-445a-83b2-6a8cdd8357fa None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:20.965342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7f527-b400-445a-83b2-6a8cdd8357fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:21.041665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3de7f527-b400-445a-83b2-6a8cdd8357fa None None] MFA Rules not processed for user `9355f8f80a434ab8af3145f7e82c277c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:21.042913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3de7f527-b400-445a-83b2-6a8cdd8357fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:21.118140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 690/1395] 10.222.0.50 () {58 vars in 1064 bytes} [Mon Aug 8 14:24:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:21.126159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714ae5f-fcde-4238-adfe-c6fd7cad6eb3 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:21.127703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f714ae5f-fcde-4238-adfe-c6fd7cad6eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:21.142492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:21.236959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714ae5f-fcde-4238-adfe-c6fd7cad6eb3 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:21.311341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:21.311943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:21.312219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:21.312520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:21.314073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:21.382486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:21.425217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:21.426650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b7c386-5afe-4035-8c85-d686294b9b4f service glance] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:21.444255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 706/1396] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:22.192605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb465-f8b8-4b27-aadf-9338f924228c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:22.193232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb465-f8b8-4b27-aadf-9338f924228c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:22.193232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb465-f8b8-4b27-aadf-9338f924228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:22.260959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-663cb465-f8b8-4b27-aadf-9338f924228c None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:22.262204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-663cb465-f8b8-4b27-aadf-9338f924228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:22.272261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:22.352299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 691/1397] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:24:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:22.396128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88efaf3-dabd-4392-8049-cca02a832311 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:22.396628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88efaf3-dabd-4392-8049-cca02a832311 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:22.396723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88efaf3-dabd-4392-8049-cca02a832311 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:22.398075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 707/1398] 10.222.0.50 () {54 vars in 943 bytes} [Mon Aug 8 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:22.405371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:22.405663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:22.405944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:22.496350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] MFA Rules not processed for user `94cf8e7d52a3441196ac4b4a63fb13ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:22.497579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:22.555666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7b28b-7bbd-46c0-b7cb-2cf31d9c2b28 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:22.555954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7b28b-7bbd-46c0-b7cb-2cf31d9c2b28 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:22.556234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7b28b-7bbd-46c0-b7cb-2cf31d9c2b28 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:22.557744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 708/1399] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:22.562426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:22.562720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:22.562999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:22.583889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:22.610094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 692/1400] 10.222.0.50 () {58 vars in 1040 bytes} [Mon Aug 8 14:24:22 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:22.619596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:22.629123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff95f79-6341-4e35-94d5-0887378c7271 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:22.630131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:22.631383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:22.739239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 709/1401] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:24:22.742711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:22.743421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:22.743696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:22.743972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:22.745609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:22.780642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:22.781857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:22.810585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:22.840855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:22.842493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77057416-ef06-4152-af7f-6cea70550ab7 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:22.850585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 693/1402] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:24:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 14:24:22.906642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525e5c8-2def-42cd-93e7-8fa2e14f2a49 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:23.048501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:23.050869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:23.051197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:23.051472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:23.053156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:23.072467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:23.156852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:23.158482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0850e6-a31f-4822-a305-1138e10d8262 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:23.179313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 710/1403] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:24:25.320442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acec740-d1c7-41b5-b60d-9f5c0451745e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:25.321024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acec740-d1c7-41b5-b60d-9f5c0451745e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:25.321024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acec740-d1c7-41b5-b60d-9f5c0451745e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:25.322460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 694/1404] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:25.327951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c546f-260a-4c39-9b03-fc6ca9246e11 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:25.328768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c546f-260a-4c39-9b03-fc6ca9246e11 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:25.329032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c546f-260a-4c39-9b03-fc6ca9246e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:25.409030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a93c546f-260a-4c39-9b03-fc6ca9246e11 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:25.410358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a93c546f-260a-4c39-9b03-fc6ca9246e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:25.513049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:25.520346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 711/1405] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:25.568687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acec740-d1c7-41b5-b60d-9f5c0451745e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:25.570197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8acec740-d1c7-41b5-b60d-9f5c0451745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:25.583254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:25.742620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acec740-d1c7-41b5-b60d-9f5c0451745e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:25.844653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:25.882786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:25.883434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:25.883719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:25.883988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:25.885653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:25.984253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:25.986337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9ce45-ef07-40df-a6a6-4b9677bd4810 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:26.008959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 695/1406] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:28.077442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc28b8-88e5-4b83-b8ea-421d069254b2 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:28.078006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc28b8-88e5-4b83-b8ea-421d069254b2 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:28.078006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc28b8-88e5-4b83-b8ea-421d069254b2 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:28.079635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 712/1407] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:28.084662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed5ae0-952e-4b00-8cc9-b1991bea2da7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:28.085101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed5ae0-952e-4b00-8cc9-b1991bea2da7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:28.085393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed5ae0-952e-4b00-8cc9-b1991bea2da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:28.159561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f6ed5ae0-952e-4b00-8cc9-b1991bea2da7 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:28.160865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f6ed5ae0-952e-4b00-8cc9-b1991bea2da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:28.211027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:28.272209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 696/1408] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:24:28.317298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc28b8-88e5-4b83-b8ea-421d069254b2 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:28.318908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-75bc28b8-88e5-4b83-b8ea-421d069254b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:28.488081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc28b8-88e5-4b83-b8ea-421d069254b2 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:28.578233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:28.675808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:28.676616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:28.676998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:28.677638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:28.679766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:28.801216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:28.802759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc6381f-e731-4095-a5bb-53d866644abc service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:28.822545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 713/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:38.269275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=23 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:38.820399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=64 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:45.376356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b4789-74bc-4abd-85ef-346c5ad12b16 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:45.376930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b4789-74bc-4abd-85ef-346c5ad12b16 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:45.376930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567b4789-74bc-4abd-85ef-346c5ad12b16 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:45.378303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 697/1410] 10.222.0.50 () {54 vars in 918 bytes} [Mon Aug 8 14:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:45.383648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bff47-b4f8-4d12-b7d2-f5d770a237af None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:45.384097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bff47-b4f8-4d12-b7d2-f5d770a237af None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:45.384189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bff47-b4f8-4d12-b7d2-f5d770a237af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:45.470580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-7e2bff47-b4f8-4d12-b7d2-f5d770a237af None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:45.471854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7e2bff47-b4f8-4d12-b7d2-f5d770a237af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:45.587042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 714/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Aug 8 14:24:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:45.607667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145990b-b07f-42ef-9135-19799f2ac64e None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:45.607969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145990b-b07f-42ef-9135-19799f2ac64e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:45.608207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145990b-b07f-42ef-9135-19799f2ac64e None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:45.609623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 698/1412] 10.222.0.50 () {54 vars in 918 bytes} [Mon Aug 8 14:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:45.614943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f181f-d643-4cf9-8b0a-c729685157ea None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:45.615268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f181f-d643-4cf9-8b0a-c729685157ea None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:45.615546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f181f-d643-4cf9-8b0a-c729685157ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:45.695936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:45.700432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3a0f181f-d643-4cf9-8b0a-c729685157ea None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:45.702738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3a0f181f-d643-4cf9-8b0a-c729685157ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:45.845196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 715/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Mon Aug 8 14:24:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:48.399490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f829f0-7bd7-4edf-aaae-da92a57e5ee9 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:48.400111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f829f0-7bd7-4edf-aaae-da92a57e5ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:48.400111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f829f0-7bd7-4edf-aaae-da92a57e5ee9 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:48.401458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:48.407211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d794-e8e6-4f4e-b59b-b9314bd4950b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:48.407731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d794-e8e6-4f4e-b59b-b9314bd4950b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:48.407863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d794-e8e6-4f4e-b59b-b9314bd4950b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:48.479786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fa68d794-e8e6-4f4e-b59b-b9314bd4950b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:48.481013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fa68d794-e8e6-4f4e-b59b-b9314bd4950b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:48.550316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:48.605974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:48.659874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f829f0-7bd7-4edf-aaae-da92a57e5ee9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:48.661472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-01f829f0-7bd7-4edf-aaae-da92a57e5ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:48.842180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f829f0-7bd7-4edf-aaae-da92a57e5ee9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:48.918243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:48.959949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:48.960626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:48.960901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:48.961170 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:48.962939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:49.053133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:49.054755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64e988-3a95-4c99-a0ab-8c6978bb86bc service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:49.072784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 700/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:24:49.996768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8531bad-1124-414f-b936-5d13e8a83594 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:49.997381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8531bad-1124-414f-b936-5d13e8a83594 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:49.997381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8531bad-1124-414f-b936-5d13e8a83594 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:49.998677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:50.004809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a526367-5bb4-48cf-a420-a41cfd0a8066 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:50.005372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a526367-5bb4-48cf-a420-a41cfd0a8066 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:50.005372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a526367-5bb4-48cf-a420-a41cfd0a8066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:50.103219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8a526367-5bb4-48cf-a420-a41cfd0a8066 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:50.105832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8a526367-5bb4-48cf-a420-a41cfd0a8066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:50.153844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:50.225965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:50.510670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8531bad-1124-414f-b936-5d13e8a83594 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:50.512176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d8531bad-1124-414f-b936-5d13e8a83594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:50.670866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8531bad-1124-414f-b936-5d13e8a83594 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:50.713876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:50.797693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:50.798316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:50.798639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:50.798958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:50.800769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:50.900611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:50.902797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a13936-d76f-4440-a57a-a05bf1379cb8 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:50.921682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 718/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:51.808321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba64e-4a49-43dc-924e-adb0b98cd039 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:51.808864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba64e-4a49-43dc-924e-adb0b98cd039 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:51.808864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ba64e-4a49-43dc-924e-adb0b98cd039 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:51.810207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:51.815192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d80f7-f9ab-48d6-858c-f2bf8b3186cf None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:51.815596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d80f7-f9ab-48d6-858c-f2bf8b3186cf None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:51.815743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d80f7-f9ab-48d6-858c-f2bf8b3186cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:51.875451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:51.899430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-812d80f7-f9ab-48d6-858c-f2bf8b3186cf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:51.900639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-812d80f7-f9ab-48d6-858c-f2bf8b3186cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:52.005085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:52.157483 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba64e-4a49-43dc-924e-adb0b98cd039 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:52.159014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9e5ba64e-4a49-43dc-924e-adb0b98cd039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:52.310370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ba64e-4a49-43dc-924e-adb0b98cd039 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:52.323115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:52.426920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:52.427458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:52.427740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:52.428015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:52.429625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:52.523463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:52.525119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8c913-296b-44e1-a1ae-39eafecdbc10 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:52.546980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 703/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:53.449945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644d684-c077-422e-9fb2-e87a00e2ef3c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:53.450552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644d684-c077-422e-9fb2-e87a00e2ef3c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:53.450552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644d684-c077-422e-9fb2-e87a00e2ef3c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:53.451880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:53.457524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754040-7d4c-4331-9075-b35407d154b4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:53.458060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754040-7d4c-4331-9075-b35407d154b4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:53.458242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754040-7d4c-4331-9075-b35407d154b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:53.481283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:53.663461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c6754040-7d4c-4331-9075-b35407d154b4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:53.664668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c6754040-7d4c-4331-9075-b35407d154b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:53.771627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:53.932386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644d684-c077-422e-9fb2-e87a00e2ef3c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:53.933952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8644d684-c077-422e-9fb2-e87a00e2ef3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:54.047564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:54.082283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644d684-c077-422e-9fb2-e87a00e2ef3c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:54.196317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:54.196917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:54.197191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:54.197459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:54.199078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:54.274824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:54.290053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:54.291886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c0ac2-e489-4daa-9daf-a1d48da8d038 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:54.311338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 721/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:55.192605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d9fc0-ac77-4989-82ca-d7a78105480d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:55.193125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d9fc0-ac77-4989-82ca-d7a78105480d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:55.193125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d9fc0-ac77-4989-82ca-d7a78105480d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:55.194389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:55.199844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2d32c-6669-408d-98da-51da0565aa48 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:55.205874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2d32c-6669-408d-98da-51da0565aa48 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:55.205874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2d32c-6669-408d-98da-51da0565aa48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:55.282634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-60a2d32c-6669-408d-98da-51da0565aa48 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:55.283899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-60a2d32c-6669-408d-98da-51da0565aa48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:55.387480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:55.538184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d9fc0-ac77-4989-82ca-d7a78105480d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:55.539868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-386d9fc0-ac77-4989-82ca-d7a78105480d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:55.629978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:55.695063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d9fc0-ac77-4989-82ca-d7a78105480d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:55.816478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:55.817000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:55.817278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:55.817556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:55.819217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:55.880271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:55.924187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:55.925817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6239295-2596-4115-b5ac-ec0c6aad4f49 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:55.944782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 706/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:56.771123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2bff1-6569-4b71-b2d0-4c190c059ce2 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:56.771734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2bff1-6569-4b71-b2d0-4c190c059ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:56.771734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2bff1-6569-4b71-b2d0-4c190c059ce2 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:56.773062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:56.778733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47df87-7c8c-48d7-99aa-802bf8e5c7c4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:56.779322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47df87-7c8c-48d7-99aa-802bf8e5c7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:56.779322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47df87-7c8c-48d7-99aa-802bf8e5c7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:56.860559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1e47df87-7c8c-48d7-99aa-802bf8e5c7c4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:56.861946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1e47df87-7c8c-48d7-99aa-802bf8e5c7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:56.962410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:57.111728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2bff1-6569-4b71-b2d0-4c190c059ce2 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:57.113309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a6e2bff1-6569-4b71-b2d0-4c190c059ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:57.161173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:57.258995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2bff1-6569-4b71-b2d0-4c190c059ce2 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:57.376091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:57.376618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:57.376895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:57.377167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:57.378760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:57.386800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:57.468671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:57.470270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a940c9-62c6-4ec9-a520-8ddcfb6e2e09 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:57.489267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 724/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:58.298492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5658bc-d68e-43e7-88a6-89068d48a83d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:58.298924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5658bc-d68e-43e7-88a6-89068d48a83d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:58.299024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5658bc-d68e-43e7-88a6-89068d48a83d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:58.300452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:58.305731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c251903-7f5c-4880-b4f0-4de222b92368 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:58.306165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c251903-7f5c-4880-b4f0-4de222b92368 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:58.306277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c251903-7f5c-4880-b4f0-4de222b92368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:58.390241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-6c251903-7f5c-4880-b4f0-4de222b92368 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:58.391540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6c251903-7f5c-4880-b4f0-4de222b92368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:58.462582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:58.500694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:24:58.652977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5658bc-d68e-43e7-88a6-89068d48a83d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:24:58.655218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6e5658bc-d68e-43e7-88a6-89068d48a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:58.675083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:58.807650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5658bc-d68e-43e7-88a6-89068d48a83d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:24:58.921748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:24:58.951476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:24:58.952091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:58.952369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:58.952644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:58.954321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:24:59.057313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:24:59.059561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2293b7-0ae0-4250-bb70-5eb58766f25b service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:24:59.077405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 709/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:24:59.886762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3683217-fbb4-4fd9-92b0-163a461bc42c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:59.887325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3683217-fbb4-4fd9-92b0-163a461bc42c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:59.887325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3683217-fbb4-4fd9-92b0-163a461bc42c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:59.888684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:24:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:24:59.893911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc38d-244b-4d66-bf95-e94d6c270697 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:24:59.894377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc38d-244b-4d66-bf95-e94d6c270697 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:24:59.894480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc38d-244b-4d66-bf95-e94d6c270697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:24:59.978591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f09bc38d-244b-4d66-bf95-e94d6c270697 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:24:59.979844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f09bc38d-244b-4d66-bf95-e94d6c270697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:00.022916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:00.080958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:00.266418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3683217-fbb4-4fd9-92b0-163a461bc42c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:00.267908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f3683217-fbb4-4fd9-92b0-163a461bc42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:00.420384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3683217-fbb4-4fd9-92b0-163a461bc42c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:00.477031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:00.537881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:00.538408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:00.538853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:00.539167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:00.540794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:00.633114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:00.634726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9ddec-9da9-4019-bfee-96c3c5efaa4f service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:00.653502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 727/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:01.477918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ce06-6324-4140-8dda-6fd226bbc0e3 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:01.478449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ce06-6324-4140-8dda-6fd226bbc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:01.478449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ce06-6324-4140-8dda-6fd226bbc0e3 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:01.480463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:01.486252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d9771-c14f-419d-95c6-a6ede18230d7 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:01.486798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d9771-c14f-419d-95c6-a6ede18230d7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:01.486886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d9771-c14f-419d-95c6-a6ede18230d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:01.571931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4c4d9771-c14f-419d-95c6-a6ede18230d7 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:01.573178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4c4d9771-c14f-419d-95c6-a6ede18230d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:01.575409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:01.676665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:01.829125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ce06-6324-4140-8dda-6fd226bbc0e3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:01.830652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5538ce06-6324-4140-8dda-6fd226bbc0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:02.005450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ce06-6324-4140-8dda-6fd226bbc0e3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:02.035328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:02.127802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:02.128420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:02.128691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:02.128961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:02.130608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:02.232799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:02.234443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba30bd-c291-43b6-a5a0-ea548d293726 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:02.254601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 712/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:04.211913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a9064-ee6c-4546-ba8a-fb71c4d5bdf1 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:04.212475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a9064-ee6c-4546-ba8a-fb71c4d5bdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:04.212475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a9064-ee6c-4546-ba8a-fb71c4d5bdf1 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:04.213999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:04.219756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9eed0-d310-446e-b926-7a558ac69717 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:04.220563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9eed0-d310-446e-b926-7a558ac69717 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:04.220563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec9eed0-d310-446e-b926-7a558ac69717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:04.263660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:04.292256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5ec9eed0-d310-446e-b926-7a558ac69717 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:04.293502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5ec9eed0-d310-446e-b926-7a558ac69717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:04.396511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:25:04.436455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a9064-ee6c-4546-ba8a-fb71c4d5bdf1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:04.438043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1e6a9064-ee6c-4546-ba8a-fb71c4d5bdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:04.548673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a9064-ee6c-4546-ba8a-fb71c4d5bdf1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:04.584716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:04.628481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:04.629091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:04.629401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:04.629673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:04.631326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:04.751214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:04.752594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931138d-6f6d-418e-89a7-6e35b666027b service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:04.773104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 730/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:06.987079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f85a2a-2605-4af7-ac43-05b3d62495a0 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:06.987682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f85a2a-2605-4af7-ac43-05b3d62495a0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:06.987682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f85a2a-2605-4af7-ac43-05b3d62495a0 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:06.989043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:06.994626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04da2-8993-4a99-8abd-132c5c9b9597 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:06.995742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04da2-8993-4a99-8abd-132c5c9b9597 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:06.995856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da04da2-8993-4a99-8abd-132c5c9b9597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:07.071024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-7da04da2-8993-4a99-8abd-132c5c9b9597 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:07.072307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7da04da2-8993-4a99-8abd-132c5c9b9597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:07.086412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:07.179324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:07.218667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f85a2a-2605-4af7-ac43-05b3d62495a0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:07.220374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a3f85a2a-2605-4af7-ac43-05b3d62495a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:07.335518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f85a2a-2605-4af7-ac43-05b3d62495a0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:07.349756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:07.413326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:07.413934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:07.414212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:07.414483 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:07.416127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:07.546876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:07.548273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94709302-f4f8-4c3e-b942-5d0e1531a5d8 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:07.568135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 715/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:09.954113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4754c-bb93-44b6-90e1-bf8a917c550f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:09.954722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4754c-bb93-44b6-90e1-bf8a917c550f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:09.954722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4754c-bb93-44b6-90e1-bf8a917c550f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:09.956034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:09.961174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b16e-3a76-4050-ba4a-edbe2c71797b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:09.961556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b16e-3a76-4050-ba4a-edbe2c71797b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:09.961729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b16e-3a76-4050-ba4a-edbe2c71797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:10.010774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:10.029579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8361b16e-3a76-4050-ba4a-edbe2c71797b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:10.030786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8361b16e-3a76-4050-ba4a-edbe2c71797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:10.126570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:10.164676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4754c-bb93-44b6-90e1-bf8a917c550f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:10.166215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-16b4754c-bb93-44b6-90e1-bf8a917c550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:10.271670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4754c-bb93-44b6-90e1-bf8a917c550f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:10.312492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:10.347002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:10.347647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:10.347928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:10.348204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:10.349944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:10.459406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:10.460782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9cc59-c178-428c-9136-b524e4f9dedb service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:10.480493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 733/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:12.636261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51b92-b2f1-4e63-98ea-bcb77341dcc5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:12.636829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51b92-b2f1-4e63-98ea-bcb77341dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:12.636829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b51b92-b2f1-4e63-98ea-bcb77341dcc5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:12.638127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:12.643514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729462ca-bc80-4e4b-afb3-4e3047b92758 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:12.643933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729462ca-bc80-4e4b-afb3-4e3047b92758 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:12.644070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729462ca-bc80-4e4b-afb3-4e3047b92758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:12.711336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-729462ca-bc80-4e4b-afb3-4e3047b92758 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:12.712562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-729462ca-bc80-4e4b-afb3-4e3047b92758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:12.734709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:12.819755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:12.860331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51b92-b2f1-4e63-98ea-bcb77341dcc5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:12.861845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b7b51b92-b2f1-4e63-98ea-bcb77341dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:12.968304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b51b92-b2f1-4e63-98ea-bcb77341dcc5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:12.988460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:13.042062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:13.042660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:13.042936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:13.043233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:13.044813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:13.156864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:13.158309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c03e8a-a354-41d7-89ad-c11797816783 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:13.176483 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 718/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:15.241737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097cb2-e5dd-4328-8b62-97b765ac117c None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:15.242278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097cb2-e5dd-4328-8b62-97b765ac117c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:15.242278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42097cb2-e5dd-4328-8b62-97b765ac117c None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:15.243614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:15.248776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb55da-864b-4c5a-a6e2-c7fe207bed22 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:15.249169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb55da-864b-4c5a-a6e2-c7fe207bed22 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:15.249338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb55da-864b-4c5a-a6e2-c7fe207bed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:15.324604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:15.327684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-60fb55da-864b-4c5a-a6e2-c7fe207bed22 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:15.329615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-60fb55da-864b-4c5a-a6e2-c7fe207bed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:15.429516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:15.468287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42097cb2-e5dd-4328-8b62-97b765ac117c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:15.469854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-42097cb2-e5dd-4328-8b62-97b765ac117c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:15.575512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42097cb2-e5dd-4328-8b62-97b765ac117c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:15.624929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:15.650570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:15.651212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:15.651497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:15.651776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:15.653389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:15.762604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:15.764078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b807d-b296-4497-915f-7dd27304910d service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:15.786088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 736/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:17.878489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3b55f-88dc-493b-836f-3e7c9b55f38d None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:17.878489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3b55f-88dc-493b-836f-3e7c9b55f38d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:17.878959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3b55f-88dc-493b-836f-3e7c9b55f38d None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:17.880053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:17.885468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fdd09c-1865-4e3e-9fd2-dcc1f5e68bc0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:17.885929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fdd09c-1865-4e3e-9fd2-dcc1f5e68bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:17.886044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fdd09c-1865-4e3e-9fd2-dcc1f5e68bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:17.953191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-75fdd09c-1865-4e3e-9fd2-dcc1f5e68bc0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:17.954436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-75fdd09c-1865-4e3e-9fd2-dcc1f5e68bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:17.984490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:18.061940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:18.099830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3b55f-88dc-493b-836f-3e7c9b55f38d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:18.101386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-17f3b55f-88dc-493b-836f-3e7c9b55f38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:18.205888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3b55f-88dc-493b-836f-3e7c9b55f38d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:18.235407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:18.280537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:18.281137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:18.281416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:18.281690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:18.283346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:18.391260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:18.392695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f79d5-dc9c-437e-9b2b-a901bcc5fd67 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:18.410836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 721/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:20.432841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851a71-0bb1-471a-b6a8-e123fccea5ff None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:20.433370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851a71-0bb1-471a-b6a8-e123fccea5ff None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:20.433370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2851a71-0bb1-471a-b6a8-e123fccea5ff None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:20.434716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:20.440141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9e3a-aad0-42c1-b25a-46133dddbe45 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:20.440557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9e3a-aad0-42c1-b25a-46133dddbe45 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:20.440698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9e3a-aad0-42c1-b25a-46133dddbe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:20.505937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-580b9e3a-aad0-42c1-b25a-46133dddbe45 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:20.507211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-580b9e3a-aad0-42c1-b25a-46133dddbe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:20.515929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:20.605338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:20.643156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851a71-0bb1-471a-b6a8-e123fccea5ff None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:20.644741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e2851a71-0bb1-471a-b6a8-e123fccea5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:20.748918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2851a71-0bb1-471a-b6a8-e123fccea5ff None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:20.805935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:20.824801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:20.825405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:20.825687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:20.825954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:20.827589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:20.938249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:20.939714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb0490-3c83-4c5f-9ba4-bf3414876266 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:20.957986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 739/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:23.044253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54e54-2884-41c2-8bcc-3ade187e8c48 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:23.044833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54e54-2884-41c2-8bcc-3ade187e8c48 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:23.044833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54e54-2884-41c2-8bcc-3ade187e8c48 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:23.046116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:23.051742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817957-97f4-4e32-b8df-9399375b573a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:23.052157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817957-97f4-4e32-b8df-9399375b573a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:23.052272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93817957-97f4-4e32-b8df-9399375b573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:23.117534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-93817957-97f4-4e32-b8df-9399375b573a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:23.118741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-93817957-97f4-4e32-b8df-9399375b573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:23.158098 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:23.217442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:23.254694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee54e54-2884-41c2-8bcc-3ade187e8c48 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:23.256277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bee54e54-2884-41c2-8bcc-3ade187e8c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:23.358954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee54e54-2884-41c2-8bcc-3ade187e8c48 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:23.397338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:23.433835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:23.434351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:23.434625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:23.434893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:23.436526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:23.543882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:23.545309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c31853-bba7-4c3b-a5a7-e13db182161f service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:23.563751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 724/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:25.602956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ff7e-1a0c-4a09-b638-7b4f343cdc3b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:25.603572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ff7e-1a0c-4a09-b638-7b4f343cdc3b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:25.603572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ff7e-1a0c-4a09-b638-7b4f343cdc3b None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:25.604963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:25.610129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928fcc9-5cfe-48e0-baa9-ef887a88f1d8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:25.610519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928fcc9-5cfe-48e0-baa9-ef887a88f1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:25.610686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928fcc9-5cfe-48e0-baa9-ef887a88f1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:25.676774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7928fcc9-5cfe-48e0-baa9-ef887a88f1d8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:25.677996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7928fcc9-5cfe-48e0-baa9-ef887a88f1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:25.696616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:25.775223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:25.813678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ff7e-1a0c-4a09-b638-7b4f343cdc3b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:25.815317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-5ac4ff7e-1a0c-4a09-b638-7b4f343cdc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:25.918651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ff7e-1a0c-4a09-b638-7b4f343cdc3b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:25.991461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:25.997906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:25.998518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:25.998801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:25.999078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:26.000729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:26.111791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:26.113228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2294892b-2d8e-4e57-bb98-389c69a677a8 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:26.132808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 742/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:25:28.137561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5328db-0f62-486f-bfdd-56c736cc8005 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:28.138135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5328db-0f62-486f-bfdd-56c736cc8005 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:28.138135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5328db-0f62-486f-bfdd-56c736cc8005 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:28.139541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:25:28.144870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868a0a4-ecb1-40ea-91b1-9bfee4bd49fd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:28.145352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868a0a4-ecb1-40ea-91b1-9bfee4bd49fd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:28.145461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868a0a4-ecb1-40ea-91b1-9bfee4bd49fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:28.210867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1868a0a4-ecb1-40ea-91b1-9bfee4bd49fd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:28.212113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1868a0a4-ecb1-40ea-91b1-9bfee4bd49fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:28.268457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:28.317651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:28.355351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5328db-0f62-486f-bfdd-56c736cc8005 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:28.356905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ee5328db-0f62-486f-bfdd-56c736cc8005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:28.460222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5328db-0f62-486f-bfdd-56c736cc8005 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:28.507147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:28.533498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:28.534116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:28.534381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:28.534655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:28.536333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:28.645225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:28.649177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80164b7-1d43-451d-a174-089aee0ebbbf service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:28.666055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 727/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:38.316296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=22 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:38.663754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=66 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:48.179573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e546-a852-4576-a761-b384eaf94e6b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:48.180105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e546-a852-4576-a761-b384eaf94e6b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:48.180105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e546-a852-4576-a761-b384eaf94e6b None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:48.181448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 744/1471] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:48.187221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce01f34-0986-4114-9b56-0f7c4e2da8c4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:48.187664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce01f34-0986-4114-9b56-0f7c4e2da8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:48.187806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce01f34-0986-4114-9b56-0f7c4e2da8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:48.254385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dce01f34-0986-4114-9b56-0f7c4e2da8c4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:48.255701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dce01f34-0986-4114-9b56-0f7c4e2da8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:48.351576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 728/1472] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:48.436322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9e473-22da-4c72-acb0-0ba6f2bfe55e None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:48.436611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9e473-22da-4c72-acb0-0ba6f2bfe55e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:48.436890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9e473-22da-4c72-acb0-0ba6f2bfe55e None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:48.438241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 745/1473] 10.222.0.50 () {54 vars in 967 bytes} [Mon Aug 8 14:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:48.443300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903bd2c-2c47-4f75-acb7-1a129e44e7e2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:48.443587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903bd2c-2c47-4f75-acb7-1a129e44e7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:48.443863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903bd2c-2c47-4f75-acb7-1a129e44e7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:48.519272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1903bd2c-2c47-4f75-acb7-1a129e44e7e2 None None] MFA Rules not processed for user `9355f8f80a434ab8af3145f7e82c277c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:48.520497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1903bd2c-2c47-4f75-acb7-1a129e44e7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:48.540781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:48.597872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 729/1474] 10.222.0.50 () {58 vars in 1064 bytes} [Mon Aug 8 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:48.605941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9e473-22da-4c72-acb0-0ba6f2bfe55e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:48.607588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-00b9e473-22da-4c72-acb0-0ba6f2bfe55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:48.725586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9e473-22da-4c72-acb0-0ba6f2bfe55e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:48.798945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:48.799242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:48.799506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:48.799780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:48.801393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:48.855316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:48.912596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:48.914025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274cea6-a7c5-45e8-8809-d2c7b24f648d service glance] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:48.934379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 746/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:51.026339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0b273-0314-4bf5-af63-9f9515fa2699 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:51.026903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0b273-0314-4bf5-af63-9f9515fa2699 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:51.026903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0b273-0314-4bf5-af63-9f9515fa2699 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:51.028261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 730/1476] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:51.033414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb9b15-3dd7-4928-b9a1-186a704273cf None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:51.033872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb9b15-3dd7-4928-b9a1-186a704273cf None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:51.033981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb9b15-3dd7-4928-b9a1-186a704273cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:51.220711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-79cb9b15-3dd7-4928-b9a1-186a704273cf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:51.221832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-79cb9b15-3dd7-4928-b9a1-186a704273cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:51.318923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 747/1477] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:51.341248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0b273-0314-4bf5-af63-9f9515fa2699 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:51.342772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-01c0b273-0314-4bf5-af63-9f9515fa2699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:51.467449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0b273-0314-4bf5-af63-9f9515fa2699 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:51.518361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:51.563707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:51.564311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:51.564591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:51.564862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:51.565499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:51.567255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:25:51.567255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:25:51.567255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:25:51.568397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3fff4-9889-4d55-ac7e-783147b1115b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:51.586758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 731/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Mon Aug 8 14:25:51 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:25:53.586670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf856f-054f-4d40-be23-8e6885ba2312 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:53.587264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf856f-054f-4d40-be23-8e6885ba2312 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:53.587264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf856f-054f-4d40-be23-8e6885ba2312 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:53.588623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 748/1479] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:53.593735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df527cb-1ec2-4995-9a37-bd026c5762f6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:53.594161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df527cb-1ec2-4995-9a37-bd026c5762f6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:53.594301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df527cb-1ec2-4995-9a37-bd026c5762f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:53.660767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9df527cb-1ec2-4995-9a37-bd026c5762f6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:53.662011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9df527cb-1ec2-4995-9a37-bd026c5762f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:53.759806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 732/1480] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:53.802772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf856f-054f-4d40-be23-8e6885ba2312 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:53.804357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1dcf856f-054f-4d40-be23-8e6885ba2312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:53.814831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:53.956983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf856f-054f-4d40-be23-8e6885ba2312 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:54.039509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:54.072662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:54.073270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:54.073550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:54.073834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:54.075454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:54.164573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:54.166090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a06a0-023f-40ab-8601-5360fd6d98e8 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:54.184676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 749/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:56.375761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0242a9-f629-44ba-a3a8-2fb5ae6f0ebb None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:56.376318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0242a9-f629-44ba-a3a8-2fb5ae6f0ebb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:56.376318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0242a9-f629-44ba-a3a8-2fb5ae6f0ebb None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:56.377640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 733/1482] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:56.382806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb0892-4000-43ef-aad7-763714f9cb74 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:56.383264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb0892-4000-43ef-aad7-763714f9cb74 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:56.383368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb0892-4000-43ef-aad7-763714f9cb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:56.449023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fceb0892-4000-43ef-aad7-763714f9cb74 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:56.450209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fceb0892-4000-43ef-aad7-763714f9cb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:56.497408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:56.548947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 750/1483] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:56.585324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0242a9-f629-44ba-a3a8-2fb5ae6f0ebb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:56.586860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bc0242a9-f629-44ba-a3a8-2fb5ae6f0ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:56.610861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:56.700047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0242a9-f629-44ba-a3a8-2fb5ae6f0ebb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:56.771280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:56.771877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:56.772157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:56.772424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:56.774003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:56.848734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:56.883615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:56.885054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4758-afa4-46ef-bf37-f22c1d4ac442 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:56.903843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 734/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:25:58.945190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce99ba-9517-448c-ab91-84006b0a0ff9 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:58.945723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce99ba-9517-448c-ab91-84006b0a0ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:58.945723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce99ba-9517-448c-ab91-84006b0a0ff9 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:58.947144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 751/1485] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:25:58.952223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6ad57-036f-4a6b-9f34-8e97a28054fe None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:58.952661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6ad57-036f-4a6b-9f34-8e97a28054fe None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:58.952767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6ad57-036f-4a6b-9f34-8e97a28054fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:59.018167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-74d6ad57-036f-4a6b-9f34-8e97a28054fe None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:25:59.019412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-74d6ad57-036f-4a6b-9f34-8e97a28054fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:59.114845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 735/1486] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:25:59.153125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce99ba-9517-448c-ab91-84006b0a0ff9 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:25:59.154732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-05ce99ba-9517-448c-ab91-84006b0a0ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:59.257332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce99ba-9517-448c-ab91-84006b0a0ff9 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:25:59.331180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:25:59.332514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:25:59.333220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:25:59.333505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:25:59.333786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:25:59.336026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:25:59.445709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:25:59.447177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fcb108-7fbf-46e6-881e-5179594d6d7e service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:25:59.466810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 752/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:26:01.662898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa0d7-4ccf-4fca-a8e1-b54f5548e0a2 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:26:01.663493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa0d7-4ccf-4fca-a8e1-b54f5548e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:26:01.663493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eaa0d7-4ccf-4fca-a8e1-b54f5548e0a2 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:26:01.664986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 736/1488] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:26:01.670283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca32a5-31e2-4dfb-9c53-3b500fd8af99 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:26:01.670704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca32a5-31e2-4dfb-9c53-3b500fd8af99 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:26:01.670815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca32a5-31e2-4dfb-9c53-3b500fd8af99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:26:01.737290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d3ca32a5-31e2-4dfb-9c53-3b500fd8af99 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:26:01.738642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d3ca32a5-31e2-4dfb-9c53-3b500fd8af99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:26:01.793863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:26:01.838522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 753/1489] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:26:01.879286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa0d7-4ccf-4fca-a8e1-b54f5548e0a2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:26:01.880841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-83eaa0d7-4ccf-4fca-a8e1-b54f5548e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:26:01.916021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:26:01.985502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eaa0d7-4ccf-4fca-a8e1-b54f5548e0a2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:26:02.059080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:26:02.059709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:26:02.059984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:26:02.060259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:26:02.061846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:26:02.141908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:26:02.171206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:26:02.172842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c7b11-eece-4e7d-abe1-54fab20cfdd6 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:26:02.190581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 737/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:26:04.427449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017b795-96f9-4b60-9cf7-d779deccc2a4 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:26:04.427971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017b795-96f9-4b60-9cf7-d779deccc2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:26:04.427971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017b795-96f9-4b60-9cf7-d779deccc2a4 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:26:04.429271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 754/1491] 10.222.0.50 () {54 vars in 927 bytes} [Mon Aug 8 14:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:26:04.434672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624909e-33e0-4d80-b00d-f6f095c1ed10 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:26:04.435135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624909e-33e0-4d80-b00d-f6f095c1ed10 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:26:04.435264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624909e-33e0-4d80-b00d-f6f095c1ed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:26:04.500117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d624909e-33e0-4d80-b00d-f6f095c1ed10 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:26:04.501335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d624909e-33e0-4d80-b00d-f6f095c1ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:26:04.598535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 738/1492] 10.222.0.50 () {58 vars in 1024 bytes} [Mon Aug 8 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:26:04.639140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017b795-96f9-4b60-9cf7-d779deccc2a4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:26:04.640768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a017b795-96f9-4b60-9cf7-d779deccc2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:26:04.745864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017b795-96f9-4b60-9cf7-d779deccc2a4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:26:04.819632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:26:04.820240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:26:04.820514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:26:04.820779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:26:04.822366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:26:04.831944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:26:04.932792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:26:04.934228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66acc4-7e1d-4e32-a80e-a14c1ceac0a0 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:26:04.952444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 755/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:26:14.596833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=82 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:26:14.950170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=50 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:21.421506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:21.422104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:21.422104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:21.497601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:27:21.498866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:21.593013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 739/1494] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:27:21.601078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:21.601564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:21.601678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:21.677646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:27:21.678765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:21.771488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 756/1495] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:27:21.776669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:21.777872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:21.789222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70e771-2576-40ef-a6e7-5c0241cccd19 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:21.851599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:21.886672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:21.887314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:21.887592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:21.887858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:21.888495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:27:21.888798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:21.889810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:21.889810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:27:21.889810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:27:21.890684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:21.903335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 740/1496] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:27:21.911606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:21.921394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c55ca5-c439-4810-83b2-928c5fc963e1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:21.983064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:22.016511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:22.017116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.017390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.017656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.018281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:22.020051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:22.020051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:27:22.020051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:27:22.021237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:22.079002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 757/1497] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:27:21 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:27:22.083372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:22.084976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:22.118816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab639519-2da2-4baa-8830-2e65c809d423 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:22.215914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:22.216523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.216803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.217102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.217764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:22.218783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:22.218783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:27:22.218783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:27:22.220021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:22.261971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 741/1498] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:27:22 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:27:22.266205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:22.267746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:22.330843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2f7a7-662c-4427-884f-082f4dfd48ac admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:22.415259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:22.427558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:22.428157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.428432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.428696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.429359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:22.430367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:22.430367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:27:22.430367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:27:22.431270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:22.443912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 758/1499] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:27:22.448637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:22.450179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:22.461522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f2ec-d156-4834-8f12-c2b894578f02 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:22.486098 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:22.556074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:22.556589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.556865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.557626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.558263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:22.559306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:22.559306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:27:22.559306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:27:22.560195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432ad2-0d92-4e5f-b608-432345f5a22d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:22.572358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 742/1500] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:27:22.577359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:22.578881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:22.612013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059cf2c-1f66-4696-9c64-cd8f6e53423a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:22.708108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:22.708762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.709041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.709308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] PATH_INFO: `/v3/projects/ea7c5adf57c44e66bf9cfc74eb8f4266/users/14470d42aa55456d8081c2e32d22d169/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.731562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea7c5adf57c44e66bf9cfc74eb8f4266, user_id=14470d42aa55456d8081c2e32d22d169, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:22.732597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:22.732597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:27:22.732597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:27:22.734552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487cba1-c285-4199-b355-af5d467d0b0f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:22.761058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 759/1501] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:27:22 2022] PUT /identity/v3/projects/ea7c5adf57c44e66bf9cfc74eb8f4266/users/14470d42aa55456d8081c2e32d22d169/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:27:22.773141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d9c5d-047a-4654-b7f9-14777a24fb62 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.773433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d9c5d-047a-4654-b7f9-14777a24fb62 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.773714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d9c5d-047a-4654-b7f9-14777a24fb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.834185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c26d9c5d-047a-4654-b7f9-14777a24fb62 None None] MFA Rules not processed for user `14470d42aa55456d8081c2e32d22d169`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:27:22.835440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c26d9c5d-047a-4654-b7f9-14777a24fb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:22.913108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 743/1502] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:27:22.926952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a38ff-958c-4af3-b13b-9bf344ef4525 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.927262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a38ff-958c-4af3-b13b-9bf344ef4525 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.927535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a38ff-958c-4af3-b13b-9bf344ef4525 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.928851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 760/1503] 10.222.0.50 () {54 vars in 964 bytes} [Mon Aug 8 14:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:27:22.934455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25725b95-4016-4461-8ca8-917d456443a6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:22.934740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25725b95-4016-4461-8ca8-917d456443a6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:22.935014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25725b95-4016-4461-8ca8-917d456443a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:22.966785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:23.011718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-25725b95-4016-4461-8ca8-917d456443a6 None None] MFA Rules not processed for user `d22e2d936518407fba2d0be379598ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:27:23.012946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25725b95-4016-4461-8ca8-917d456443a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:23.089187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 744/1504] 10.222.0.50 () {58 vars in 1061 bytes} [Mon Aug 8 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:27:23.096893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a38ff-958c-4af3-b13b-9bf344ef4525 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:23.098391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7e1a38ff-958c-4af3-b13b-9bf344ef4525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:23.148656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:23.205445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a38ff-958c-4af3-b13b-9bf344ef4525 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:23.279876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:23.280479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:23.280757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:23.281039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:23.282704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:23.380499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:23.381852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:23.385302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:23.399976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 761/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:27:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:27:23.445505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eadfe1-5c18-473d-918c-4265cbe7cae5 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:23.445800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eadfe1-5c18-473d-918c-4265cbe7cae5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:23.446078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eadfe1-5c18-473d-918c-4265cbe7cae5 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:23.447373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 745/1506] 10.222.0.50 () {52 vars in 830 bytes} [Mon Aug 8 14:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:27:23.450916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:23.452161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:23.552128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eb7da-d84f-4c62-aa0f-35bc884ca7d8 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:23.644753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:23.645430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:23.645529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:23.645796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] PATH_INFO: `/v3/users/14470d42aa55456d8081c2e32d22d169` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:23.658254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14470d42aa55456d8081c2e32d22d169)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:23.659303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:23.659303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:27:23.659303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:27:23.660488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:23.669917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:23.728039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14470d42aa55456d8081c2e32d22d169'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:23.737487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14470d42aa55456d8081c2e32d22d169'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:23.737966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14470d42aa55456d8081c2e32d22d169'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:23.752008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14470d42aa55456d8081c2e32d22d169'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:23.774709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14470d42aa55456d8081c2e32d22d169'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:23.781003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 762/1507] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:27:23 2022] DELETE /identity/v3/users/14470d42aa55456d8081c2e32d22d169 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:27:23.785415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:23.785915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:23.786013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:23.843326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:27:23.844544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:23.921295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:23.935226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 746/1508] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:27:23.944699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:23.945940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:24.044012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37a61b-97c4-4e08-811f-e69b9cc76825 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:24.098550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:24.119017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:24.120160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:24.120438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:24.120707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:24.122313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:24.232426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:24.233788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158c3d0-6226-412c-b3be-c177c803b6a2 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:24.252664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 763/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:27:24.599982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:27:24.601306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:27:24.613302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fc5bf-2278-422d-9096-c4ab8a8922e1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:27:24.712507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:27:24.713135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:27:24.713464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:27:24.713733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] PATH_INFO: `/v3/projects/ea7c5adf57c44e66bf9cfc74eb8f4266` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:27:24.723595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea7c5adf57c44e66bf9cfc74eb8f4266)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:27:24.724625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:27:24.724625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:27:24.724625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:27:24.725820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:27:24.782835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea7c5adf57c44e66bf9cfc74eb8f4266', 'request_id': 'req-a4d1b39f-e184-4e57-80d3-270844655625'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:24.788794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea7c5adf57c44e66bf9cfc74eb8f4266', 'request_id': 'req-a4d1b39f-e184-4e57-80d3-270844655625'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:24.804222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea7c5adf57c44e66bf9cfc74eb8f4266', 'request_id': 'req-a4d1b39f-e184-4e57-80d3-270844655625'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:24.831750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] The token cache is being invalidate because project ea7c5adf57c44e66bf9cfc74eb8f4266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:27:24.832043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:24.832474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a4d1b39f-e184-4e57-80d3-270844655625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:27:24.833858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 747/1510] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:27:24 2022] DELETE /identity/v3/projects/ea7c5adf57c44e66bf9cfc74eb8f4266 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:27:34.250353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=62 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:27:34.829826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:28:53.510583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3316ab-bcf7-4494-85d1-185d0b5b94ee None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:53.511246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3316ab-bcf7-4494-85d1-185d0b5b94ee None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:53.511246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3316ab-bcf7-4494-85d1-185d0b5b94ee None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:53.512548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:28:53.517319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d008-1a04-42e2-9eb2-414d3fe96192 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:53.517723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d008-1a04-42e2-9eb2-414d3fe96192 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:53.517883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d008-1a04-42e2-9eb2-414d3fe96192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:53.598670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0679d008-1a04-42e2-9eb2-414d3fe96192 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:28:53.599933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0679d008-1a04-42e2-9eb2-414d3fe96192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:28:53.702881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:28:53.861236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3316ab-bcf7-4494-85d1-185d0b5b94ee None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:28:53.863427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7b3316ab-bcf7-4494-85d1-185d0b5b94ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:28:53.981117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3316ab-bcf7-4494-85d1-185d0b5b94ee None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:28:53.987242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f19c3d1-83fc-4da7-a9ef-3340644460b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:28:53.987843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19c3d1-83fc-4da7-a9ef-3340644460b1 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:53.988119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19c3d1-83fc-4da7-a9ef-3340644460b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:53.988389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f19c3d1-83fc-4da7-a9ef-3340644460b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:53.996310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19c3d1-83fc-4da7-a9ef-3340644460b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:28:53.997876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f19c3d1-83fc-4da7-a9ef-3340644460b1 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:28:54.007019 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 765/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:28:54.840926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a378-7f06-459c-9814-1615f5d6dacc None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:54.841656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a378-7f06-459c-9814-1615f5d6dacc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:54.841656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61a378-7f06-459c-9814-1615f5d6dacc None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:54.842811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:28:54.848043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13daa0b-1d7d-4d0a-bc07-ad62f047643d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:54.849679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13daa0b-1d7d-4d0a-bc07-ad62f047643d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:54.849679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13daa0b-1d7d-4d0a-bc07-ad62f047643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:54.868891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f13daa0b-1d7d-4d0a-bc07-ad62f047643d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:28:54.870095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f13daa0b-1d7d-4d0a-bc07-ad62f047643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:28:54.878309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:28:55.033442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a378-7f06-459c-9814-1615f5d6dacc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:28:55.048962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61a378-7f06-459c-9814-1615f5d6dacc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:28:55.054513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b03fa-0b1e-4bf2-ae07-c3ed1d211ccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:28:55.055139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b03fa-0b1e-4bf2-ae07-c3ed1d211ccd service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:55.055423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b03fa-0b1e-4bf2-ae07-c3ed1d211ccd service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:55.055698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b03fa-0b1e-4bf2-ae07-c3ed1d211ccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:55.064804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b03fa-0b1e-4bf2-ae07-c3ed1d211ccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:28:55.066430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b03fa-0b1e-4bf2-ae07-c3ed1d211ccd service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:28:55.076263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 750/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:28:55.550251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbbd6d-6dc4-495c-ada6-14e5d51aed0b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:55.550478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbbd6d-6dc4-495c-ada6-14e5d51aed0b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:55.550750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbbd6d-6dc4-495c-ada6-14e5d51aed0b None None] PATH_INFO: `/v3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:55.552093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 767/1517] 10.222.0.16 () {54 vars in 925 bytes} [Mon Aug 8 14:28:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:28:55.558107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db2278-f4c1-46c0-a8b0-22fd17774714 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:55.558397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db2278-f4c1-46c0-a8b0-22fd17774714 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:55.558683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db2278-f4c1-46c0-a8b0-22fd17774714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:55.619216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b7db2278-f4c1-46c0-a8b0-22fd17774714 None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:28:55.620417 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b7db2278-f4c1-46c0-a8b0-22fd17774714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:28:55.637526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:28:55.692874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 751/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 8 14:28:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:28:59.465756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7e94-f844-48ba-9912-8606331f85f0 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:59.466291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7e94-f844-48ba-9912-8606331f85f0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:59.466291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b7e94-f844-48ba-9912-8606331f85f0 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:59.467637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 768/1519] 10.222.0.50 () {54 vars in 961 bytes} [Mon Aug 8 14:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:28:59.472426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdc852-10f0-4a03-99ca-09be184e7ae2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:59.472847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdc852-10f0-4a03-99ca-09be184e7ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:59.472985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdc852-10f0-4a03-99ca-09be184e7ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:59.492366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-89bdc852-10f0-4a03-99ca-09be184e7ae2 None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:28:59.493548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-89bdc852-10f0-4a03-99ca-09be184e7ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:28:59.502655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 752/1520] 10.222.0.50 () {58 vars in 1058 bytes} [Mon Aug 8 14:28:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:28:59.509934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7e94-f844-48ba-9912-8606331f85f0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:28:59.523259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b7e94-f844-48ba-9912-8606331f85f0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:28:59.528277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79effa-e873-4a68-91a9-411343072ac1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:28:59.528870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79effa-e873-4a68-91a9-411343072ac1 service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:28:59.529150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79effa-e873-4a68-91a9-411343072ac1 service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:28:59.529437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79effa-e873-4a68-91a9-411343072ac1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:28:59.537734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79effa-e873-4a68-91a9-411343072ac1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:28:59.539476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79effa-e873-4a68-91a9-411343072ac1 service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:28:59.549633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 769/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:28:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:29:06.293890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb7545-cdc1-47ff-9abc-b37dff4bac07 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:06.294444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb7545-cdc1-47ff-9abc-b37dff4bac07 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:06.294444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb7545-cdc1-47ff-9abc-b37dff4bac07 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:06.295973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:29:06.301923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c24ee-69bf-4957-aff8-bfb2ebbe561e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:06.302415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c24ee-69bf-4957-aff8-bfb2ebbe561e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:06.302513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c24ee-69bf-4957-aff8-bfb2ebbe561e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:06.321998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-468c24ee-69bf-4957-aff8-bfb2ebbe561e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:29:06.323231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-468c24ee-69bf-4957-aff8-bfb2ebbe561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:06.331116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:29:06.487663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb7545-cdc1-47ff-9abc-b37dff4bac07 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:29:06.502312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb7545-cdc1-47ff-9abc-b37dff4bac07 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:29:06.508223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648baeb4-f2a6-4e7b-b05f-48aa1086dcb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:29:06.508820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648baeb4-f2a6-4e7b-b05f-48aa1086dcb9 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:06.509098 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648baeb4-f2a6-4e7b-b05f-48aa1086dcb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:06.509371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648baeb4-f2a6-4e7b-b05f-48aa1086dcb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:06.517557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648baeb4-f2a6-4e7b-b05f-48aa1086dcb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:29:06.519171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648baeb4-f2a6-4e7b-b05f-48aa1086dcb9 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:29:06.528548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 754/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:29:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:29:07.354997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e8963-021b-41a1-af7e-7d297b3d3430 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:07.355485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e8963-021b-41a1-af7e-7d297b3d3430 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:07.355569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e8963-021b-41a1-af7e-7d297b3d3430 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:07.357030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 14:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:29:07.363147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc00e-8106-48b8-881c-9e3440380a6f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:07.363649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc00e-8106-48b8-881c-9e3440380a6f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:07.363748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc00e-8106-48b8-881c-9e3440380a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:07.383018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0b4dc00e-8106-48b8-881c-9e3440380a6f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:29:07.384291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0b4dc00e-8106-48b8-881c-9e3440380a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:07.392934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 14:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:29:07.545695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e8963-021b-41a1-af7e-7d297b3d3430 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:29:07.560708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e8963-021b-41a1-af7e-7d297b3d3430 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:29:07.566925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f72c81-fc2e-4965-8983-a45e1e37df8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:29:07.567546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72c81-fc2e-4965-8983-a45e1e37df8b service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:07.567822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72c81-fc2e-4965-8983-a45e1e37df8b service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:07.568091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f72c81-fc2e-4965-8983-a45e1e37df8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:07.576140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72c81-fc2e-4965-8983-a45e1e37df8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:29:07.577704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f72c81-fc2e-4965-8983-a45e1e37df8b service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:29:07.586939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 772/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:29:11.418190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92dc-22d7-401f-9e39-a3c852031e2a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:11.418758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92dc-22d7-401f-9e39-a3c852031e2a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:11.418758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c92dc-22d7-401f-9e39-a3c852031e2a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:11.420156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:29:11.426025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f7d3-4a98-491b-8ddb-3f182b755be8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:11.426454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f7d3-4a98-491b-8ddb-3f182b755be8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:11.426571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20f7d3-4a98-491b-8ddb-3f182b755be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:11.446006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ee20f7d3-4a98-491b-8ddb-3f182b755be8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:29:11.447219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ee20f7d3-4a98-491b-8ddb-3f182b755be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:11.455404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:29:11.529162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c92dc-22d7-401f-9e39-a3c852031e2a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:29:11.530921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ca5c92dc-22d7-401f-9e39-a3c852031e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:11.608868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5c92dc-22d7-401f-9e39-a3c852031e2a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:29:11.612998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab10a5-3db0-466a-8378-2f51b3d395f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:29:11.613600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab10a5-3db0-466a-8378-2f51b3d395f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:11.613882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab10a5-3db0-466a-8378-2f51b3d395f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:11.614156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab10a5-3db0-466a-8378-2f51b3d395f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:11.623460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab10a5-3db0-466a-8378-2f51b3d395f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:29:11.624900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab10a5-3db0-466a-8378-2f51b3d395f2 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:29:11.634717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 757/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:29:13.603140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f833c-2b99-4dca-a239-7e8b6f34bc46 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:13.603721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f833c-2b99-4dca-a239-7e8b6f34bc46 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:13.603721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f833c-2b99-4dca-a239-7e8b6f34bc46 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:13.605014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:29:13.610100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aba39-babf-4560-a6f8-3e0d3653443c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:13.610505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aba39-babf-4560-a6f8-3e0d3653443c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:13.610659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aba39-babf-4560-a6f8-3e0d3653443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:13.629788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b82aba39-babf-4560-a6f8-3e0d3653443c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:29:13.630998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b82aba39-babf-4560-a6f8-3e0d3653443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:13.638788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:29:13.674825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f833c-2b99-4dca-a239-7e8b6f34bc46 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:29:13.685343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f833c-2b99-4dca-a239-7e8b6f34bc46 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:29:13.688825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab72ddb-1ff0-43c7-bcc8-2a4c895e79f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:29:13.689404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72ddb-1ff0-43c7-bcc8-2a4c895e79f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:13.689678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72ddb-1ff0-43c7-bcc8-2a4c895e79f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:13.689948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab72ddb-1ff0-43c7-bcc8-2a4c895e79f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:13.698609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72ddb-1ff0-43c7-bcc8-2a4c895e79f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:29:13.700044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab72ddb-1ff0-43c7-bcc8-2a4c895e79f6 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:29:13.709702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 775/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:29:17.218538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3ee7c-1095-460e-b15a-dfb300634166 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:17.219069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3ee7c-1095-460e-b15a-dfb300634166 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:17.219069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3ee7c-1095-460e-b15a-dfb300634166 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:17.220393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:29:17.226218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347b01e-ac1f-4c98-8981-4be806aa2255 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:17.226657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347b01e-ac1f-4c98-8981-4be806aa2255 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:17.226767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347b01e-ac1f-4c98-8981-4be806aa2255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:17.234413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=81 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:29:17.247598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4347b01e-ac1f-4c98-8981-4be806aa2255 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:29:17.249095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4347b01e-ac1f-4c98-8981-4be806aa2255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:17.256951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:29:17.565365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3ee7c-1095-460e-b15a-dfb300634166 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:29:17.567056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10b3ee7c-1095-460e-b15a-dfb300634166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:17.631355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:29:17.645364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3ee7c-1095-460e-b15a-dfb300634166 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:29:17.648992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4c9e75-0ca9-4237-8bd8-67f24b75ddac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:29:17.649590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c9e75-0ca9-4237-8bd8-67f24b75ddac service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:17.649868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c9e75-0ca9-4237-8bd8-67f24b75ddac service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:17.650141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4c9e75-0ca9-4237-8bd8-67f24b75ddac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:17.659054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c9e75-0ca9-4237-8bd8-67f24b75ddac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:29:17.660515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4c9e75-0ca9-4237-8bd8-67f24b75ddac service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:29:17.669906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 760/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:29:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:29:19.686234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb041-cab6-4077-bec3-72c89c5bb499 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:19.686806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb041-cab6-4077-bec3-72c89c5bb499 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:19.686806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cb041-cab6-4077-bec3-72c89c5bb499 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:19.688164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 14:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:29:19.693944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b0e19-0377-481b-9ab3-358abd844cae None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:19.694380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b0e19-0377-481b-9ab3-358abd844cae None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:19.694380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b0e19-0377-481b-9ab3-358abd844cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:19.714052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-130b0e19-0377-481b-9ab3-358abd844cae None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:29:19.715294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-130b0e19-0377-481b-9ab3-358abd844cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:19.723789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 14:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:29:20.024221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb041-cab6-4077-bec3-72c89c5bb499 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:29:20.025931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-925cb041-cab6-4077-bec3-72c89c5bb499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:29:20.039112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cb041-cab6-4077-bec3-72c89c5bb499 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:29:20.042634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396ef27-4b0f-4833-b6ed-47bcae4c4b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:29:20.043253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ef27-4b0f-4833-b6ed-47bcae4c4b94 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:29:20.043538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ef27-4b0f-4833-b6ed-47bcae4c4b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:29:20.043809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ef27-4b0f-4833-b6ed-47bcae4c4b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:29:20.052761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396ef27-4b0f-4833-b6ed-47bcae4c4b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:29:20.054179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396ef27-4b0f-4833-b6ed-47bcae4c4b94 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:29:20.063434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 778/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:29:29.709356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=11 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:29:30.059435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=8 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:41.560021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:41.560684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:41.560684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:41.586583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:41.587834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:41.600921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 762/1540] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:30:41.609966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:41.611651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:41.611651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:41.635489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:41.636706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:41.652030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 779/1541] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:41.656651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:41.667650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e2626-836a-4433-85f9-16486dbd0cbc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:41.674896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:41.675717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:41.675994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:41.676263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:41.676915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:41.677217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:41.679845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:41.694364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 763/1542] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:41.703417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:41.716769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce35953-bf04-41db-9444-1aca75577cb0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:41.729152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:41.729802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:41.730087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:41.730361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:41.731147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:41.733329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:41.829995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 780/1543] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:41 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:41.837195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:41.854748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991eea84-068e-472c-9d47-93968b20d6e2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:41.976868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:41.977499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:41.977779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:41.978051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:41.978698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:41.980688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.019458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 764/1544] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:41 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:42.026067 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.039785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521beb9f-ba5b-4707-889f-6b79ede38567 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.052535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6897-e855-414c-a106-0de5d4f744ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.053155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6897-e855-414c-a106-0de5d4f744ff admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.053559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6897-e855-414c-a106-0de5d4f744ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.053836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6897-e855-414c-a106-0de5d4f744ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.059149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6897-e855-414c-a106-0de5d4f744ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.059149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6897-e855-414c-a106-0de5d4f744ff admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.072085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 781/1545] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:42.076248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.092418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277c490-d257-4c19-92fd-75bf65b0a12c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.099767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.100450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.100789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.101129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.103083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.104067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.109361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9184c-bb12-493a-8959-b5f0332d9fd1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.109684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9184c-bb12-493a-8959-b5f0332d9fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.109976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9184c-bb12-493a-8959-b5f0332d9fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.128124 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 765/1546] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:42.135110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.150237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d2b9184c-bb12-493a-8959-b5f0332d9fd1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.151521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d2b9184c-bb12-493a-8959-b5f0332d9fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.161682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d672b-55c5-4e79-8160-1312ceab5efc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.170047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 782/1547] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:30:42.180209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.180498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.180782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.188529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.189345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.189725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.190096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] PATH_INFO: `/v3/projects/f542c80a8d1e4e86b2e3cfe2dc1ead8b/users/4c9812784d464aaea4e7862f173b5f4c/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.214839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.224342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.229052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f542c80a8d1e4e86b2e3cfe2dc1ead8b, user_id=4c9812784d464aaea4e7862f173b5f4c, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.230219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:30:42.230219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:30:42.230219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:30:42.232482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd8190-f1c7-4e6a-93a1-74ecc0452518 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.241546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 783/1548] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.248389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.265971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5402a7-9e19-42a5-b6d2-eabdcc93b985 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.276666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.277288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.277576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.279232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.279232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.283615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.284968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:30:42.284968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:30:42.284968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:30:42.286140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafd94c-86da-4296-b728-eda2dadac1dd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.289256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 766/1549] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:42 2022] PUT /identity/v3/projects/f542c80a8d1e4e86b2e3cfe2dc1ead8b/users/4c9812784d464aaea4e7862f173b5f4c/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:42.294414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d093-9a77-4dcb-9066-7b96f51ec5bc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.294812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d093-9a77-4dcb-9066-7b96f51ec5bc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.296721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54d093-9a77-4dcb-9066-7b96f51ec5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.313681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 784/1550] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.315122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3d8e-4433-416d-bd3b-1e21226e2f7b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.317532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3d8e-4433-416d-bd3b-1e21226e2f7b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.317532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3d8e-4433-416d-bd3b-1e21226e2f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.338464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6e54d093-9a77-4dcb-9066-7b96f51ec5bc None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.340164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6e54d093-9a77-4dcb-9066-7b96f51ec5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.343180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c9af3d8e-4433-416d-bd3b-1e21226e2f7b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.344308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c9af3d8e-4433-416d-bd3b-1e21226e2f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.415882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 767/1551] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.417217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.417501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.417790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.421814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 785/1552] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.423388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb88a78-9012-41e0-9525-68e18c094a3b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.423883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb88a78-9012-41e0-9525-68e18c094a3b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.424156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb88a78-9012-41e0-9525-68e18c094a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.443634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3fb88a78-9012-41e0-9525-68e18c094a3b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.444645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.444808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3fb88a78-9012-41e0-9525-68e18c094a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.445891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.459455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 786/1553] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.459455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80297e8a-3af7-40cb-b937-050f9ad21ae4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.459455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80297e8a-3af7-40cb-b937-050f9ad21ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.459824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 768/1554] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.459933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80297e8a-3af7-40cb-b937-050f9ad21ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.460021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.466598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc7380-55c1-45e7-b090-08da2c4b4019 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.471751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.472351 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.472633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.472904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.473578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.474611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:30:42.474611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:30:42.474611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:30:42.475830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21de4a31-3da0-4c3f-96eb-a576f2f92cc1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.485574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-80297e8a-3af7-40cb-b937-050f9ad21ae4 None None] MFA Rules not processed for user `4c9812784d464aaea4e7862f173b5f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.486772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-80297e8a-3af7-40cb-b937-050f9ad21ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.509808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 769/1555] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:42.511064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ce6d-6777-4ec8-9871-e375caebec0b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.511383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ce6d-6777-4ec8-9871-e375caebec0b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.511656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8ce6d-6777-4ec8-9871-e375caebec0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.527733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:42.535345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3fd8ce6d-6777-4ec8-9871-e375caebec0b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.537151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3fd8ce6d-6777-4ec8-9871-e375caebec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.558204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 787/1556] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.560129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8bcea-5129-433e-9786-ffb50e0cc9d3 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.560421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8bcea-5129-433e-9786-ffb50e0cc9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.560693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8bcea-5129-433e-9786-ffb50e0cc9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.579394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fbd8bcea-5129-433e-9786-ffb50e0cc9d3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.580588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fbd8bcea-5129-433e-9786-ffb50e0cc9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.585145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:42.600319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 770/1557] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.602202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cdc2c-8e48-44a8-8bf4-950c3f4a968a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.602488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cdc2c-8e48-44a8-8bf4-950c3f4a968a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.602781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9cdc2c-8e48-44a8-8bf4-950c3f4a968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.622653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6b9cdc2c-8e48-44a8-8bf4-950c3f4a968a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.624003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6b9cdc2c-8e48-44a8-8bf4-950c3f4a968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.632786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 771/1558] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.633974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56ca7-9dc4-466b-a60d-5ef313ec5e2d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.634254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56ca7-9dc4-466b-a60d-5ef313ec5e2d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.634526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf56ca7-9dc4-466b-a60d-5ef313ec5e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.653958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9bf56ca7-9dc4-466b-a60d-5ef313ec5e2d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.655168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9bf56ca7-9dc4-466b-a60d-5ef313ec5e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.659055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 788/1559] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.660326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.660617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.660891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.668308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 772/1560] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.669527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.669808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.670080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.685096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.686310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.688625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.689784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.694257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 789/1561] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.695043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.698052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 773/1562] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.699030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.707991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeade24-7fdc-4a09-9dc4-b91046aa4968 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.708276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7547fe-a493-4b03-b5c6-de317c414340 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.712978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.713376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.713574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.713851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.713969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.714119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.714269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.714512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.714735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.715160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.715800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:30:42.715800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:30:42.715800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:30:42.716972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb9e12-28aa-43b5-8ca9-ac5a1b6679f4 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.717091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.739567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 790/1563] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/users => generated 294 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:42.740817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2547-c7c8-4817-ad9d-f37ecd0e673b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.741100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2547-c7c8-4817-ad9d-f37ecd0e673b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.741370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2547-c7c8-4817-ad9d-f37ecd0e673b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.750471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 774/1564] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:42.750875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.761432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-47ea2547-c7c8-4817-ad9d-f37ecd0e673b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.762696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-47ea2547-c7c8-4817-ad9d-f37ecd0e673b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.817858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa387a2-943a-4278-9c2e-d35fe9aeccec admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.822945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.823704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.823991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.824270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.824926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.825235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.826722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 791/1565] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.826871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.827981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.828271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.828552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.842108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 775/1566] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.842990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.849035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:42.850562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:42.853782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef334c81-f1fd-4d28-be2e-6f3248f53db9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.858514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.859304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 792/1567] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:42.859445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.859445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.859734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.859966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.860369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.860671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.862292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.869656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f369f-3845-4027-83eb-5e08118a77da None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.875623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 776/1568] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.876918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.878168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.878810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.879106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.879384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.880038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.880340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.882080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.888162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c801542-9920-4041-be2e-f3b74389c4ca admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.892808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.893426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.893903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.894027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.900629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 793/1569] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.900808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.900808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.901007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.901114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.905282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7db33-6daa-472d-893b-a3dc04624369 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.909785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.910381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.910647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.910911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.911582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.920200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 777/1570] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.920392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.920748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.927770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 794/1571] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:42.927770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.927987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991ba5b-7915-4658-aed7-87eb26e236ed admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.931993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.932678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.932960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.933243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.933887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.935382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7300a-71e4-4c1d-83d9-5244c9f7d7c3 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.935861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.939938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.940590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.940950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.941277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.942024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.942385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.944045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.956653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 795/1572] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.957454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.961269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 778/1573] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:42.962182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.968221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed78809-b4b0-40cb-8910-6229ee0c40fa admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.971848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d4f35-7e89-49fb-81e2-27045083550b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:42.972913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.973555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.973838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.974111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.974763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.976438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:42.976712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.977021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:42.977302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:42.977568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:42.978200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:42.978500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:42.980101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:42.992954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 779/1574] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:42.994249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:42.999694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:43.006448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f228540-15ae-44e9-9f05-135cf5ff31a9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.011931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.012594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.013041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.013386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.013534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 796/1575] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.014356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.016318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.017466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.040326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:43.047189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 780/1576] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:42 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.047189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.084851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e82b6e-5a73-4443-8102-cfa89048553f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.107752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b7aed2-ed5d-49cf-81df-8c1c4d606a0e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.112376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.112988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.113273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.113548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.114182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.116151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.141593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.142209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.142492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.142769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.143437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.144641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 781/1577] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.145360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.145583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.177861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 797/1578] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/projects => generated 403 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.177861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.210188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6ad4-f390-4fb8-abe8-5ea1d2058a8a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.241367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fe970-0080-4af0-978b-09224c82a977 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.246733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.247454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.247738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.248017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.248667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.250270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.260265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 798/1579] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.261372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.268901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.269518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.269796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.270071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.270714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.272344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.273184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a651b-a40f-41e7-ae6b-c323a5ea4b38 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.278019 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.278609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.278894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.279197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.279816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.281692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.282400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 782/1580] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.283217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.293213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0651f-d04b-40df-a06f-42007bf141a1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.298149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.298745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.299026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.299335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.299966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.301920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.307904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:43.313911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 799/1581] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.313911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.341001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 783/1582] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.341907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.350130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:43.384337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d26024-fa8a-416e-a5f8-b8fc379336f0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.405305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0a3b-ede6-424e-8265-a6bf2de6bcbc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.410298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.410914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.411239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.411527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.412164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.414708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.439972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 784/1583] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.440809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.445678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.446292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.446575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.446849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.447515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.449465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.451266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93628cc-25b8-4884-9525-ad40d635ab06 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.458537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.459534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.459937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.460225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.461542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.464052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.475313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 800/1584] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.475313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.483819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778830-f861-4251-b82a-4319d3d89d6e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.489023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.489637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.489901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.490313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.490675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 785/1585] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/users => generated 289 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.490938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.491559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.492939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.501778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408f5ec-f6ed-4290-ae37-b4154e25bbf3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.506617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.507519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.507757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.508017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] PATH_INFO: `/v3/projects/2044ed14955242bfa45c10cd3e370ffd/users/a1d38c20ccd649ecbce25c62d4b2758a/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.514814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 801/1586] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/users => generated 302 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.515743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.525892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2044ed14955242bfa45c10cd3e370ffd, user_id=a1d38c20ccd649ecbce25c62d4b2758a, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.526342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14c217-f881-452d-9ad1-a607a9fca3b7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.528824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.530963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.531646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.531928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.532199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] PATH_INFO: `/v3/projects/fa2ecaeb6faf4c1ea5eac70b19398b42/users/84d2f3c3c5fb4cceabf9a35f64a1551c/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.542482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 786/1587] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:43 2022] PUT /identity/v3/projects/2044ed14955242bfa45c10cd3e370ffd/users/a1d38c20ccd649ecbce25c62d4b2758a/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:43.543849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.549905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2ecaeb6faf4c1ea5eac70b19398b42, user_id=84d2f3c3c5fb4cceabf9a35f64a1551c, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.552747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.565962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 802/1588] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:43 2022] PUT /identity/v3/projects/fa2ecaeb6faf4c1ea5eac70b19398b42/users/84d2f3c3c5fb4cceabf9a35f64a1551c/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:43.566942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.608456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b0ec8-bcfc-4e3e-b8e9-7fd64082d9d7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.628233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced902b-fa5b-4b3e-a71c-207e44499140 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.633006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.633616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.634239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.634239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.634847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.636749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.650058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:43.659039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 803/1589] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.659039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.667905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.668517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.668797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.669136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.669764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.669881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b593f-722f-4d1e-b161-7793c524bc8b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.671987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.675253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.675877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.676161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.676433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.677103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.678685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.685847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:43.691706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 804/1590] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.692772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.697861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 787/1591] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:43.698561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.704220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137b5f2-b6fe-4078-9bc6-8746c254f95f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.708241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd581052-2af2-4059-b55f-0fbb40403292 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.708985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.709578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.709858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.710129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.711432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.713022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.713480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.714069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.714350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.714620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.715299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.716866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.723883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 805/1592] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.724663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.732443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 788/1593] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.732443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.735178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95ca9-f8b2-41b3-a5a5-ace4cc485d3c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.740222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0dc42-4923-46b1-8653-8866b992657a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.740354 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a168c59-64f4-4228-9662-d9d3fc1802c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.740844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a168c59-64f4-4228-9662-d9d3fc1802c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.741124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a168c59-64f4-4228-9662-d9d3fc1802c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.741394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a168c59-64f4-4228-9662-d9d3fc1802c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.742076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a168c59-64f4-4228-9662-d9d3fc1802c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.743698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a168c59-64f4-4228-9662-d9d3fc1802c9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.744770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.745353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.745631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.745903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.746531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.748178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.755022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 806/1594] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.756401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.756879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.757956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.759376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 789/1595] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.760178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.771457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f08686-bc30-4c6d-8a47-a2b9bb478648 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.776963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.777573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.777706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:43.777858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.778265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.778970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.779080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:43.780598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.790829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 807/1596] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:43.791687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.795041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 790/1597] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.795775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.804628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d1f88-f4fb-4531-8356-64ad1dd86fb6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.807555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e903ac-652f-4e37-9f63-efc8cb646631 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.809437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.810043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.810358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.810657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.811350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.812415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.812965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.813091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.813286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.813561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.814343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.815898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.824471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 808/1598] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.829650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 791/1599] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.829650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.829920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.838776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf0848-fda6-405b-9c05-71e57eb9bb33 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.838979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828bf15-f065-42a0-a867-e50e76ab1607 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.843269 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.843502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.843855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.844086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.844194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.844367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.844482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.844641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.845060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.845275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.846679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.846845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.857601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 792/1600] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.857791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 809/1601] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.860645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.860850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.870651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473f014-3a6c-4847-a40d-5a82837535c8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.873958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36736ad-910c-4747-9fb3-1057a1e36231 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.877907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.878642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.878777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.879044 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.879719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.880493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.881176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.881395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.881500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.881745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] PATH_INFO: `/v3/projects/2044ed14955242bfa45c10cd3e370ffd/users/a1d38c20ccd649ecbce25c62d4b2758a/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.884283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2044ed14955242bfa45c10cd3e370ffd, user_id=a1d38c20ccd649ecbce25c62d4b2758a, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.887106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.893088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 793/1602] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.893960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.899563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 810/1603] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:43 2022] PUT /identity/v3/projects/2044ed14955242bfa45c10cd3e370ffd/users/a1d38c20ccd649ecbce25c62d4b2758a/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:43.900342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.905240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8486b0-8977-4675-8235-f62b4be8edcf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.958043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e729c79-db9f-45f9-aee9-98b1910f23cf admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.962091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.962264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.962789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.962943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.963068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.963193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.963372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.963479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.964065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.965666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.972534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.973962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.975646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 794/1604] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:43.976499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.985472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 811/1605] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:43.986416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12804911-46a7-4a1b-b32d-957bb0534d13 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:43.986568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:43.991078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:43.991823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:43.992179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:43.992532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:43.993260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:43.994947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:43.996736 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dc7b0-23dd-4c40-b107-465a573c2117 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.002308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.003283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.003741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.004016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] PATH_INFO: `/v3/projects/450cacbbb551460198179c4a0c7c18ac/users/a193081e23864453987cd63da6493049/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.006837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 795/1606] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:44.011392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.021753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88fcff-643a-41ec-b5e4-ed4a92117115 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.022971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=450cacbbb551460198179c4a0c7c18ac, user_id=a193081e23864453987cd63da6493049, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.026346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.027401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.028137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.028495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.028839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] PATH_INFO: `/v3/projects/0400861a88074a1480b54ce5e651cc76/users/75b75be070a147fba4788b0f51d8d0fb/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.041249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 812/1607] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:43 2022] PUT /identity/v3/projects/450cacbbb551460198179c4a0c7c18ac/users/a193081e23864453987cd63da6493049/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:44.042403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.050479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0400861a88074a1480b54ce5e651cc76, user_id=75b75be070a147fba4788b0f51d8d0fb, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.053381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.068470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 796/1608] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:44 2022] PUT /identity/v3/projects/0400861a88074a1480b54ce5e651cc76/users/75b75be070a147fba4788b0f51d8d0fb/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:44.069587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.102899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:44.133957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6a383-2f06-4240-91b4-12315ac0cdbb admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.138924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39fa61-66ed-4d77-beb6-15b4c34eec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.139789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39fa61-66ed-4d77-beb6-15b4c34eec1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.139789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39fa61-66ed-4d77-beb6-15b4c34eec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.140089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39fa61-66ed-4d77-beb6-15b4c34eec1a admin admin] PATH_INFO: `/v3/projects/82cfe2f487fc4393bda828dc28471dca/users/33c71ba6b2ad438d8593fa19eb617a34/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.143208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc9b1f-3a48-461e-9835-ac3e6441a873 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.148142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.148802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.149136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.149413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] PATH_INFO: `/v3/projects/0dac648b55f54fa0b56b9d3fde7d0437/users/707a007eb7f34e2d83132743eedba159/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.161080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39fa61-66ed-4d77-beb6-15b4c34eec1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82cfe2f487fc4393bda828dc28471dca, user_id=33c71ba6b2ad438d8593fa19eb617a34, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.164120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39fa61-66ed-4d77-beb6-15b4c34eec1a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.167986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dac648b55f54fa0b56b9d3fde7d0437, user_id=707a007eb7f34e2d83132743eedba159, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.172682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.182800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:44.187774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 813/1609] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:44 2022] PUT /identity/v3/projects/82cfe2f487fc4393bda828dc28471dca/users/33c71ba6b2ad438d8593fa19eb617a34/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:44.189582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.189937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.190442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.195853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 797/1610] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:44 2022] PUT /identity/v3/projects/0dac648b55f54fa0b56b9d3fde7d0437/users/707a007eb7f34e2d83132743eedba159/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:44.197032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.230488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] MFA Rules not processed for user `a1d38c20ccd649ecbce25c62d4b2758a`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.231913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.279126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e79d3-4f3e-491a-83bf-de987fbd87fe admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.285459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f62f-bc83-4e45-9d20-3f0fbaa8d92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.286047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f62f-bc83-4e45-9d20-3f0fbaa8d92f admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.286333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f62f-bc83-4e45-9d20-3f0fbaa8d92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.286609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f62f-bc83-4e45-9d20-3f0fbaa8d92f admin admin] PATH_INFO: `/v3/projects/cfc86120e53145328dd430f542e73182/users/48a91f6797cc4604ba2606e0549234d0/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.303764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5f62f-bc83-4e45-9d20-3f0fbaa8d92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc86120e53145328dd430f542e73182, user_id=48a91f6797cc4604ba2606e0549234d0, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.307244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5f62f-bc83-4e45-9d20-3f0fbaa8d92f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.314095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 814/1611] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.315224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.327641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 798/1612] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:44 2022] PUT /identity/v3/projects/cfc86120e53145328dd430f542e73182/users/48a91f6797cc4604ba2606e0549234d0/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:44.332495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdb737-d267-4762-ad4d-f51e5bb1d12b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.332777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdb737-d267-4762-ad4d-f51e5bb1d12b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.333128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdb737-d267-4762-ad4d-f51e5bb1d12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.336341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd88d-a492-4369-b830-4dadb8aa1342 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.385310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0fbdb737-d267-4762-ad4d-f51e5bb1d12b None None] MFA Rules not processed for user `a193081e23864453987cd63da6493049`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.387553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0fbdb737-d267-4762-ad4d-f51e5bb1d12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.419645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a6791-a7a3-44a3-b79f-ad8a796c594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.420338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a6791-a7a3-44a3-b79f-ad8a796c594f admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.421096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a6791-a7a3-44a3-b79f-ad8a796c594f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.421585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a6791-a7a3-44a3-b79f-ad8a796c594f admin admin] PATH_INFO: `/v3/projects/bdcd265da07a488cb1b022a41d2503ae/users/bee7fd2630f94e8ab54b181c4a761535/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.447299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a6791-a7a3-44a3-b79f-ad8a796c594f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcd265da07a488cb1b022a41d2503ae, user_id=bee7fd2630f94e8ab54b181c4a761535, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.450135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a6791-a7a3-44a3-b79f-ad8a796c594f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.465740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 815/1613] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:44 2022] PUT /identity/v3/projects/bdcd265da07a488cb1b022a41d2503ae/users/bee7fd2630f94e8ab54b181c4a761535/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:44.467330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67762-75e3-479f-8e3d-9f8785b95f37 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.467750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67762-75e3-479f-8e3d-9f8785b95f37 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.468312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67762-75e3-479f-8e3d-9f8785b95f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.471660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 799/1614] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.478215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.478561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.478776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.498618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e1d67762-75e3-479f-8e3d-9f8785b95f37 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.499923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e1d67762-75e3-479f-8e3d-9f8785b95f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.511368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.512688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.532929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:44.577063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 816/1615] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.577063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba246c0-4801-4a74-83ab-295850c15c0f None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.577063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba246c0-4801-4a74-83ab-295850c15c0f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.577063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba246c0-4801-4a74-83ab-295850c15c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.591225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 800/1616] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.591582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:44.592102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.613674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0ba246c0-4801-4a74-83ab-295850c15c0f None None] MFA Rules not processed for user `707a007eb7f34e2d83132743eedba159`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.614915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0ba246c0-4801-4a74-83ab-295850c15c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.651713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c1f5f-a5cb-4468-b780-0e41080a1864 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.658954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a75f3-78fe-495c-9cc4-fbf6b81fae56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.659202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a75f3-78fe-495c-9cc4-fbf6b81fae56 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.659580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a75f3-78fe-495c-9cc4-fbf6b81fae56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.660015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a75f3-78fe-495c-9cc4-fbf6b81fae56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.699914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 817/1617] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.699914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b663c7-73bf-43e3-9f6e-4e05079d0670 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.699914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b663c7-73bf-43e3-9f6e-4e05079d0670 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.699914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b663c7-73bf-43e3-9f6e-4e05079d0670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.724020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d7b663c7-73bf-43e3-9f6e-4e05079d0670 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.725764 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d7b663c7-73bf-43e3-9f6e-4e05079d0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.732714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a75f3-78fe-495c-9cc4-fbf6b81fae56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.734669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a75f3-78fe-495c-9cc4-fbf6b81fae56 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.736459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 818/1618] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.738122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.738448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.738722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.753396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 801/1619] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:44.755072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11075ba-cbc8-47c4-87f7-40f837df4def None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.755708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11075ba-cbc8-47c4-87f7-40f837df4def None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.756256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11075ba-cbc8-47c4-87f7-40f837df4def None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.758146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 802/1620] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 14:30:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:30:44.759077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11075ba-cbc8-47c4-87f7-40f837df4def None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.761326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b11075ba-cbc8-47c4-87f7-40f837df4def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.786056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] MFA Rules not processed for user `bee7fd2630f94e8ab54b181c4a761535`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:44.788071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:44.831876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11075ba-cbc8-47c4-87f7-40f837df4def None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.836847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.837521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.837795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.838066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.849156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.850773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.863445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 803/1621] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:44.865485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.882103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06fda-ee62-4b00-8716-329300aad154 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.885706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 819/1622] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:44.890903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.891195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.891822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.892099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.892444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.903769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150fe2f-1655-4c91-bc55-a638fd1fffe3 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:44.912768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7da39-bb7f-4378-8d13-9c9c529b3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:44.913424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7da39-bb7f-4378-8d13-9c9c529b3d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.914320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7da39-bb7f-4378-8d13-9c9c529b3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.917173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7da39-bb7f-4378-8d13-9c9c529b3d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.917845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7da39-bb7f-4378-8d13-9c9c529b3d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.927447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:44.928277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7da39-bb7f-4378-8d13-9c9c529b3d4e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.932457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:44.956911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 804/1623] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:44.957773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:44.966049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:44.973279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 820/1624] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/projects => generated 405 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:44.974704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cf6c8-b8c9-48c1-add9-b0411bd77bf5 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:44.975070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cf6c8-b8c9-48c1-add9-b0411bd77bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:44.975452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cf6c8-b8c9-48c1-add9-b0411bd77bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:44.979152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a223f39-922c-4fa2-be88-61378ac73ca6 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.031480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9e6cf6c8-b8c9-48c1-add9-b0411bd77bf5 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:45.033446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9e6cf6c8-b8c9-48c1-add9-b0411bd77bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:45.040490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.042064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.042477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.042858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.106293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:45.151012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.152804 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.167436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 805/1625] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:45.168565 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.197299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 821/1626] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 14:30:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:45.198755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b892b-2aa5-4828-be79-1e8a63f6842f None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.199048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b892b-2aa5-4828-be79-1e8a63f6842f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.199358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b892b-2aa5-4828-be79-1e8a63f6842f None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.200709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 822/1627] 10.222.0.50 () {54 vars in 962 bytes} [Mon Aug 8 14:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:30:45.203891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b892b-2aa5-4828-be79-1e8a63f6842f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.234679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b892b-2aa5-4828-be79-1e8a63f6842f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.249054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.249697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.249984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.250298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.250999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.253211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.254000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca9cea-d61d-4c75-93f0-af9adfdec1eb service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.260237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2434f973-ac3d-4055-bc05-d67550315169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.260237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434f973-ac3d-4055-bc05-d67550315169 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.260521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434f973-ac3d-4055-bc05-d67550315169 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.260733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2434f973-ac3d-4055-bc05-d67550315169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.312879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 823/1628] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:45 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:45.312879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.359196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b0530-d3f5-4ebe-aa2a-1ea45549a200 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.364291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434f973-ac3d-4055-bc05-d67550315169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.367664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2434f973-ac3d-4055-bc05-d67550315169 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.375227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.377384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.378153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.378700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.385630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 806/1629] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:45.390312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.390973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.392872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.448727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:45.468703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:45.470876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:45.496991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 807/1630] 10.222.0.50 () {58 vars in 1059 bytes} [Mon Aug 8 14:30:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:45.508797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.532835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f9944-f44f-4952-a73c-6fcb14c98143 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.546984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.548786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.549054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.549323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.551264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.557438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.561993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.564915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.574379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 808/1631] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:45.575776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.582631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 824/1632] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:45.595140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.611682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d4413-958f-4e29-995c-2ea08b0d9ea2 service nova] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.617079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a62ce-1ed4-4e05-bc18-9cf4f24ca3b7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.619882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.622494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.622939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.624031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] PATH_INFO: `/v3/projects/401e068177f24171b1e7974d0a577f83/users/fbb8c499a0774fbd8c3e7f900364bde2/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.634948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.635763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.638547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.639566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.662982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401e068177f24171b1e7974d0a577f83, user_id=fbb8c499a0774fbd8c3e7f900364bde2, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.667482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.691694 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 825/1633] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:45 2022] PUT /identity/v3/projects/401e068177f24171b1e7974d0a577f83/users/fbb8c499a0774fbd8c3e7f900364bde2/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:45.696996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.772202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.776926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.805169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 809/1634] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:45.869579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045496de-874a-4546-a57e-a405650892b3 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.886596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771acbb-7f0c-4848-afdc-367b2c31766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.889595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771acbb-7f0c-4848-afdc-367b2c31766f admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.912825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771acbb-7f0c-4848-afdc-367b2c31766f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.914453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771acbb-7f0c-4848-afdc-367b2c31766f admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.918097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771acbb-7f0c-4848-afdc-367b2c31766f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:45.925016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771acbb-7f0c-4848-afdc-367b2c31766f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:45.963124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 826/1635] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:45.967367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:45.982254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52ac73-4093-4361-938e-9554884bd0f2 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:45.989789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e267d-31a0-4cf6-864a-c338c24b3862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:45.992592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e267d-31a0-4cf6-864a-c338c24b3862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:45.993310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e267d-31a0-4cf6-864a-c338c24b3862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:45.993806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e267d-31a0-4cf6-864a-c338c24b3862 admin admin] PATH_INFO: `/v3/projects/401e068177f24171b1e7974d0a577f83/users/fbb8c499a0774fbd8c3e7f900364bde2/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:45.998593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e267d-31a0-4cf6-864a-c338c24b3862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401e068177f24171b1e7974d0a577f83, user_id=fbb8c499a0774fbd8c3e7f900364bde2, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:46.004872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e267d-31a0-4cf6-864a-c338c24b3862 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:46.034115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 810/1636] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:45 2022] PUT /identity/v3/projects/401e068177f24171b1e7974d0a577f83/users/fbb8c499a0774fbd8c3e7f900364bde2/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:46.045349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451146de-5970-4809-bd52-3bc59652903b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.049379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451146de-5970-4809-bd52-3bc59652903b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.054774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451146de-5970-4809-bd52-3bc59652903b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.123621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-451146de-5970-4809-bd52-3bc59652903b None None] MFA Rules not processed for user `fbb8c499a0774fbd8c3e7f900364bde2`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:46.124848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-451146de-5970-4809-bd52-3bc59652903b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:46.136436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272015b9-f9be-45db-86c2-b1b716de6d21 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.137475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272015b9-f9be-45db-86c2-b1b716de6d21 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.137842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272015b9-f9be-45db-86c2-b1b716de6d21 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.139818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 811/1637] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 14:30:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 14:30:46.146788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.147015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.147476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.176920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:46.199217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:46.203913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:46.349145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 827/1638] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:46.349145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451146de-5970-4809-bd52-3bc59652903b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:46.384812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 812/1639] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:46.404266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:46.421994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451146de-5970-4809-bd52-3bc59652903b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:46.427378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe0c45-d667-4024-9fa6-4638434a5f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:46.427894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe0c45-d667-4024-9fa6-4638434a5f24 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.428175 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe0c45-d667-4024-9fa6-4638434a5f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.428449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe0c45-d667-4024-9fa6-4638434a5f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.439481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe0c45-d667-4024-9fa6-4638434a5f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:46.441207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe0c45-d667-4024-9fa6-4638434a5f24 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:46.465381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 828/1640] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:46.468102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563878a-f9c5-402f-bf59-618182afb235 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.472011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563878a-f9c5-402f-bf59-618182afb235 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.472343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563878a-f9c5-402f-bf59-618182afb235 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.477705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 829/1641] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 14:30:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:30:46.486867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.487216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.487571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.530537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:46.531437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:46.542900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 830/1642] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:30:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:46.589521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:46.601577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edee4f1-1b8e-48bc-805d-ed1559682cd3 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:46.606710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:46.607496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.607780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.608053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.621110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78238e-d7d9-493a-abf0-a8f3586b371e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:46.621443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:46.622820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:46.625263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a77bc-faf4-4df8-96a8-bfbef84bc570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:46.626041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a77bc-faf4-4df8-96a8-bfbef84bc570 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:46.626423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a77bc-faf4-4df8-96a8-bfbef84bc570 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:46.626792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a77bc-faf4-4df8-96a8-bfbef84bc570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:46.647764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a77bc-faf4-4df8-96a8-bfbef84bc570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:46.649397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a77bc-faf4-4df8-96a8-bfbef84bc570 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:46.662353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 831/1643] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:46.676037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 813/1644] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:47.784603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:47.972234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae806f5b-6f08-4d44-947a-da3443b8e4cc service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:47.989148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:47.990126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:47.990580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:47.991021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] PATH_INFO: `/v3/users/707a007eb7f34e2d83132743eedba159` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:47.993332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=707a007eb7f34e2d83132743eedba159)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:47.997395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:48.095225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:48.111723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707a007eb7f34e2d83132743eedba159'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:48.133667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707a007eb7f34e2d83132743eedba159'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:48.139283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707a007eb7f34e2d83132743eedba159'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:48.162925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707a007eb7f34e2d83132743eedba159'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:48.203762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707a007eb7f34e2d83132743eedba159'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:48.224756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 832/1645] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:30:47 2022] DELETE /identity/v3/users/707a007eb7f34e2d83132743eedba159 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:48.229239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:48.229816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:48.229816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:48.261184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:48.262441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:48.276006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:48.485816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 814/1646] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:48.493971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:48.501705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:48.663128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b2794-a835-4335-84f1-19d2b65b7b78 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:48.673454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:48.675191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:48.675562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:48.676678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:48.718909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:48.726404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:48.763038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 833/1647] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:50.690576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:50.693833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:50.722276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:50.723730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:50.734591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bcdaa-437e-4a8e-be01-34b9b4d39d40 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:50.765806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:50.766480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:50.766763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:50.767045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] PATH_INFO: `/v3/projects/0dac648b55f54fa0b56b9d3fde7d0437` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:50.768426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2614341-a572-4190-82f2-469742151ef3 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:50.771027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dac648b55f54fa0b56b9d3fde7d0437)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:50.777501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:50.782959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:50.785313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:50.785629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:50.785916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] PATH_INFO: `/v3/users/bee7fd2630f94e8ab54b181c4a761535` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:50.793745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee7fd2630f94e8ab54b181c4a761535)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:50.796192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:50.922613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dac648b55f54fa0b56b9d3fde7d0437', 'request_id': 'req-2b085b6b-87cd-4b4f-98db-578cfa3959fc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.942234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dac648b55f54fa0b56b9d3fde7d0437', 'request_id': 'req-2b085b6b-87cd-4b4f-98db-578cfa3959fc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.949173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee7fd2630f94e8ab54b181c4a761535'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.954250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dac648b55f54fa0b56b9d3fde7d0437', 'request_id': 'req-2b085b6b-87cd-4b4f-98db-578cfa3959fc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.972395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee7fd2630f94e8ab54b181c4a761535'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.973100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee7fd2630f94e8ab54b181c4a761535'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.986188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] The token cache is being invalidate because project 0dac648b55f54fa0b56b9d3fde7d0437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:30:50.986488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.988524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.989848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee7fd2630f94e8ab54b181c4a761535'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:50.994499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 815/1648] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:30:50 2022] DELETE /identity/v3/projects/0dac648b55f54fa0b56b9d3fde7d0437 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:50.995661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:51.000005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:51.030959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cdccba0d-160c-4bb0-8406-812199a8e06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee7fd2630f94e8ab54b181c4a761535'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:51.058027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 834/1649] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:30:50 2022] DELETE /identity/v3/users/bee7fd2630f94e8ab54b181c4a761535 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:30:51.064413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:51.064706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:51.064985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:51.116096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:51.118216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:51.197509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:51.278486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b085b6b-87cd-4b4f-98db-578cfa3959fc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:51.311567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 835/1650] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:51.348200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:51.350387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:51.385789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:51.430777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:51.431648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:51.432034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:51.432437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] PATH_INFO: `/v3/users/fbb8c499a0774fbd8c3e7f900364bde2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:51.437008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb8c499a0774fbd8c3e7f900364bde2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:51.438172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: WARNING py.warnings [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:30:51.438172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: warnings.warn(msg) Aug 08 14:30:51.438172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: Aug 08 14:30:51.442516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:51.532547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3c780-262a-46d1-bc1a-2d73007354e6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:51.538832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:51.539564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:51.539886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:51.540170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:51.553501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:51.555362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:51.568937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 836/1651] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:51.579337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb8c499a0774fbd8c3e7f900364bde2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:51.597909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb8c499a0774fbd8c3e7f900364bde2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:51.611939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb8c499a0774fbd8c3e7f900364bde2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:51.614276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb8c499a0774fbd8c3e7f900364bde2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:51.659062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7cf3122d-b5c4-4273-8fd0-97bbddfa990f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb8c499a0774fbd8c3e7f900364bde2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:51.672177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 816/1652] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:30:50 2022] DELETE /identity/v3/users/fbb8c499a0774fbd8c3e7f900364bde2 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:53.360680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:53.388880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380950b-fa20-4dba-ab8a-df762b552dcb service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:53.559547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:53.560391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:53.560805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:53.561235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] PATH_INFO: `/v3/projects/401e068177f24171b1e7974d0a577f83` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:53.563941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401e068177f24171b1e7974d0a577f83)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:53.565154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: WARNING py.warnings [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 14:30:53.565154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: warnings.warn(msg) Aug 08 14:30:53.565154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: Aug 08 14:30:53.566639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:53.675576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401e068177f24171b1e7974d0a577f83', 'request_id': 'req-d3873a94-f796-4903-bb21-b19f3c2eee01'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:53.689256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401e068177f24171b1e7974d0a577f83', 'request_id': 'req-d3873a94-f796-4903-bb21-b19f3c2eee01'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:53.710132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401e068177f24171b1e7974d0a577f83', 'request_id': 'req-d3873a94-f796-4903-bb21-b19f3c2eee01'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:53.732406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] The token cache is being invalidate because project 401e068177f24171b1e7974d0a577f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:30:53.733058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:53.735058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d3873a94-f796-4903-bb21-b19f3c2eee01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:53.740750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 837/1653] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:30:53 2022] DELETE /identity/v3/projects/401e068177f24171b1e7974d0a577f83 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:53.742931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:53.743525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:53.743644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:53.794207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:53.797213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:53.941603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:53.942365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:53.942810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:53.947480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 817/1654] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:53.954565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:53.977074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:53.979249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:53.997664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:54.005900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 838/1655] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:54.010245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b4979-5fa4-48f5-a6ed-41fce7d1b634 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:54.020458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:54.021332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:54.021826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:54.022168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:54.025335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:54.025797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:54.027859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:54.056885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 818/1656] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:54.058183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:54.073092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4090b26-13ff-49b3-8cea-cd05c3bd93f1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:54.092028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f4bac-748c-4a14-8b21-da581b4d5b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:54.092928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4bac-748c-4a14-8b21-da581b4d5b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:54.094140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4bac-748c-4a14-8b21-da581b4d5b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:54.094574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f4bac-748c-4a14-8b21-da581b4d5b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:54.095399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f4bac-748c-4a14-8b21-da581b4d5b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:54.099046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f4bac-748c-4a14-8b21-da581b4d5b54 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:54.175581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 839/1657] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:54 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:30:54.179410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:54.200687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7fb1f-0603-4d97-a774-e318d0598f62 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:54.328959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:54.354625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:54.358254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:54.360913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:54.361649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:54.362445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:54.379173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:54.384889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:54.385380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:54.388776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:54.463396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 819/1658] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:54 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:54.466751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:54.479384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:54.483549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] MFA Rules not processed for user `84d2f3c3c5fb4cceabf9a35f64a1551c`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:54.487565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:54.535340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba63d9-4d8f-46fc-a7cd-71e8752904f2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:54.558276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:54.560615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:54.562771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:54.566325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] PATH_INFO: `/v3/projects/bdcd265da07a488cb1b022a41d2503ae` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:54.571677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcd265da07a488cb1b022a41d2503ae)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:54.580802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:54.768172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 840/1659] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:54.770771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:54.794702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ca896-4658-43ff-93b7-06a096b14656 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:54.799190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcd265da07a488cb1b022a41d2503ae', 'request_id': 'req-104a157a-ac00-4bd1-9587-7cdd1da639b9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:54.817692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcd265da07a488cb1b022a41d2503ae', 'request_id': 'req-104a157a-ac00-4bd1-9587-7cdd1da639b9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:54.880468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcd265da07a488cb1b022a41d2503ae', 'request_id': 'req-104a157a-ac00-4bd1-9587-7cdd1da639b9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:54.909878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] The token cache is being invalidate because project bdcd265da07a488cb1b022a41d2503ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:30:54.911102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:54.914849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:54.919035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 820/1660] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:30:54 2022] DELETE /identity/v3/projects/bdcd265da07a488cb1b022a41d2503ae => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:54.920902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:54.925778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:55.016472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55658608-27f8-4473-8c0f-91fd106dc58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.017295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55658608-27f8-4473-8c0f-91fd106dc58b admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.018049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55658608-27f8-4473-8c0f-91fd106dc58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.018049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55658608-27f8-4473-8c0f-91fd106dc58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.021506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55658608-27f8-4473-8c0f-91fd106dc58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.026154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55658608-27f8-4473-8c0f-91fd106dc58b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.043845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 841/1661] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:55.045945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.046245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.048155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.070982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:55.072492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:55.088430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 842/1662] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:55.088430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.088430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:55.100300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9afbd-a0b1-4aa1-b93d-5d7be89cfb33 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.106097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8100c0f0-0b09-4f2a-983c-928356605e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.106887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100c0f0-0b09-4f2a-983c-928356605e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.107943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100c0f0-0b09-4f2a-983c-928356605e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.108337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8100c0f0-0b09-4f2a-983c-928356605e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.109117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100c0f0-0b09-4f2a-983c-928356605e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.110892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8100c0f0-0b09-4f2a-983c-928356605e8d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.133243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 843/1663] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:55.136125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.136729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.137302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.157533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a157a-ac00-4bd1-9587-7cdd1da639b9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.164714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.165735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.166501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:55.167188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.167691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.168012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:55.170102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:55.184465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 844/1664] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:55.184465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.198462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784680b4-aba5-4d26-b24f-0ce6a437cefa None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.204126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.205574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.205867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.206377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] PATH_INFO: `/v3/projects/76b492740950465e92d7be4d3cbb9024/users/1e77bad300054ed58c520829b0721501/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.251279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b492740950465e92d7be4d3cbb9024, user_id=1e77bad300054ed58c520829b0721501, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.262403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.283370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 845/1665] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:55 2022] PUT /identity/v3/projects/76b492740950465e92d7be4d3cbb9024/users/1e77bad300054ed58c520829b0721501/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:55.283942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.316287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:55.336578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:55.341909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.343793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cfdcf-1678-4b8e-876c-40642e58da37 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.364206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 821/1666] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:55.368212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.368212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.368212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.412317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] MFA Rules not processed for user `1e77bad300054ed58c520829b0721501`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:55.415811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:55.441887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b733686-80fa-41d5-a8b8-79d847f13837 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.456010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.456912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.460776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.461690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.462555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:30:55.463010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.465357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.494550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 846/1667] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:30:55.499637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.521878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de96f5-2642-4b36-8952-4a48c8fb5bd3 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.534529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.535185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.535467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.535738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.536562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.543833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.606583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 847/1668] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:55.610977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.633766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 822/1669] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:55.638759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.659726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40acd2-7bc7-4972-824f-806c23254890 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.767787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764f93a-6ecf-4e0a-ad81-3b180a7fbbbd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:55.789266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.789786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.789984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.790314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.790929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.796030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.801084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:55.801760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:55.802146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:55.802427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:55.803323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:55.805709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:55.869685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 823/1670] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/projects => generated 373 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:55.870217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 848/1671] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/users => generated 296 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:30:55.874402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.882815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:55.920191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbab098-61c1-4c13-a9d8-561602e8feef admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.038677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c57ef-9d2a-4b6c-abd2-71ec8fa84fc1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.066655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.068063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.068488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.071009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.071262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.071856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.071962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.072195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.072954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.073971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.075402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.079018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.098831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:56.109518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 849/1672] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:56.113065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.139773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 824/1673] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:55 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:56.139956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c9efd9-4d83-4326-83c7-645b8fb79314 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.151863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.152066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.152331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.152607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.153270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.153788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.154901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.165008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:56.185388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 850/1674] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:56.186397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.204936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be7b91-84e4-4aeb-b5b4-b0080ef4b2a2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.218496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22507d53-8c0b-449a-9bb7-0a31102d9a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.219213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22507d53-8c0b-449a-9bb7-0a31102d9a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.219516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22507d53-8c0b-449a-9bb7-0a31102d9a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.219762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22507d53-8c0b-449a-9bb7-0a31102d9a5b admin admin] PATH_INFO: `/v3/projects/d8ab4c0542ff46878576d61c3ca8c9fc/users/e456d6b0e7b9430783ff5c0a442b82c2/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.246548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b038a-f032-4b3e-a1b1-3b39b7b8b617 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.254256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22507d53-8c0b-449a-9bb7-0a31102d9a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ab4c0542ff46878576d61c3ca8c9fc, user_id=e456d6b0e7b9430783ff5c0a442b82c2, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.259359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22507d53-8c0b-449a-9bb7-0a31102d9a5b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.273100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.273736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.274015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.274283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.274933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.276580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.292916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 851/1675] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:56 2022] PUT /identity/v3/projects/d8ab4c0542ff46878576d61c3ca8c9fc/users/e456d6b0e7b9430783ff5c0a442b82c2/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:56.296852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.297116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.298479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.309463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 825/1676] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:56.313580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.330582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31affd-0800-4bd2-9599-a97e6474d37d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.349194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] MFA Rules not processed for user `e456d6b0e7b9430783ff5c0a442b82c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:56.352605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:56.430815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.431561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.431853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.432108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] PATH_INFO: `/v3/projects/3cb75c9e05684f5b90d7f4623397f9e4/users/013ce7f1b61944939f75f3506e4eab7b/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.458795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb75c9e05684f5b90d7f4623397f9e4, user_id=013ce7f1b61944939f75f3506e4eab7b, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.462604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.485988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 826/1677] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:56 2022] PUT /identity/v3/projects/3cb75c9e05684f5b90d7f4623397f9e4/users/013ce7f1b61944939f75f3506e4eab7b/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:30:56.486789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 852/1678] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:56.491880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.497765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.517971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec24ef-bed6-4d2e-b95d-ffd73a0232e0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.618758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.619706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.620407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.620972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.622047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.627254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.628194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a54d6-f70e-44d4-9c63-134ee32c5350 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.646012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.646655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.646935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.647234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.647931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.653078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.676889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 827/1679] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:56.682057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.699468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 853/1680] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:30:56 2022] POST /identity/v3/projects => generated 393 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:56.701469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.701865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843bcce-03f4-4eda-85ce-db8d0e09ace7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.828461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:56.846411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb13e37-df40-4e33-a294-107be5ee1c94 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:56.853542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.854373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.854980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.855564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.856363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.859891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.880003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bbdbe-70e2-4498-a207-4631dbfba6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:56.880717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bbdbe-70e2-4498-a207-4631dbfba6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.881002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bbdbe-70e2-4498-a207-4631dbfba6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.881277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bbdbe-70e2-4498-a207-4631dbfba6f4 admin admin] PATH_INFO: `/v3/projects/3cb75c9e05684f5b90d7f4623397f9e4/users/013ce7f1b61944939f75f3506e4eab7b/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.884816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bbdbe-70e2-4498-a207-4631dbfba6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb75c9e05684f5b90d7f4623397f9e4, user_id=013ce7f1b61944939f75f3506e4eab7b, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:56.888208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bbdbe-70e2-4498-a207-4631dbfba6f4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:56.910814 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 828/1681] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:56 2022] PUT /identity/v3/projects/3cb75c9e05684f5b90d7f4623397f9e4/users/013ce7f1b61944939f75f3506e4eab7b/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:56.918346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 854/1682] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:30:56 2022] POST /identity/v3/users => generated 303 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:30:56.920465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:56.920978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:56.921221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:56.921502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:56.938469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:56.945344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32b71a-3a14-43d8-92ae-95a5dc5fc526 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:57.003779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] MFA Rules not processed for user `013ce7f1b61944939f75f3506e4eab7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:57.003779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:57.143729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:57.145788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:57.146072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:57.146344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:57.147007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:57.155244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:57.187136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 855/1683] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:57.188954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:57.208893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb402e65-9f0f-4b4a-8231-4dd7bd4b0e4a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:57.217340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:57.218352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:57.219008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:57.219473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] PATH_INFO: `/v3/projects/b311c0cd195148d48637bfd66661ed17/users/65099867e2fd4cdc8c37421078bd57eb/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:57.231526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 829/1684] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:57.270605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b311c0cd195148d48637bfd66661ed17, user_id=65099867e2fd4cdc8c37421078bd57eb, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:57.275572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:57.277703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:57.278009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:57.305222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 856/1685] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:57 2022] PUT /identity/v3/projects/b311c0cd195148d48637bfd66661ed17/users/65099867e2fd4cdc8c37421078bd57eb/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:57.306270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:57.483048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6afb-3e5b-4bd0-ab5b-ccc667e4f7e8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:57.508362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bb69e-12bd-4770-baa6-f34ca3ce6d7d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:57.513684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:57.514313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:57.514595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:57.515633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:57.516313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:57.517961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:57.539162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 857/1686] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:30:57.547165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:57.561476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ceb44-5381-48be-beef-0abbb539f742 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:57.567862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9616ac-a39f-4655-8588-6d3cdb759682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:57.569301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9616ac-a39f-4655-8588-6d3cdb759682 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:57.569301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9616ac-a39f-4655-8588-6d3cdb759682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:57.569301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9616ac-a39f-4655-8588-6d3cdb759682 admin admin] PATH_INFO: `/v3/projects/b311c0cd195148d48637bfd66661ed17/users/65099867e2fd4cdc8c37421078bd57eb/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:57.573657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9616ac-a39f-4655-8588-6d3cdb759682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b311c0cd195148d48637bfd66661ed17, user_id=65099867e2fd4cdc8c37421078bd57eb, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:57.577954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9616ac-a39f-4655-8588-6d3cdb759682 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:57.591443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 858/1687] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:30:57 2022] PUT /identity/v3/projects/b311c0cd195148d48637bfd66661ed17/users/65099867e2fd4cdc8c37421078bd57eb/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:57.604133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:57.604454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:57.604689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:57.662045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] MFA Rules not processed for user `65099867e2fd4cdc8c37421078bd57eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:57.664716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:57.682718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:57.684735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:57.685037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:57.686055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:57.698552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:57.773599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 859/1688] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:57 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:57.884596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:57.896261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:57.898160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:57.936256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 830/1689] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:30:58.001154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:58.004325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:58.077527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:58.080333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:58.112458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aeddf-951a-4f61-ae9e-d7179b42b47c service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:58.191114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568d3e3-d379-4a36-b62a-d596545fc83f None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:58.200536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:58.201296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:58.201768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:58.202048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:58.238004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:58.239516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:58.244049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:58.245258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:58.245732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:58.246201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] PATH_INFO: `/v3/users/a193081e23864453987cd63da6493049` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:58.248104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a193081e23864453987cd63da6493049)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:58.250330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:58.266891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 860/1690] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:57 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:58.346951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:58.366013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aecc95-5bbf-4b00-851b-4901f08932f7 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:58.386924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:58.389337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:58.389630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:58.389910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:58.420878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a193081e23864453987cd63da6493049'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:58.431247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a193081e23864453987cd63da6493049'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:58.442873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a193081e23864453987cd63da6493049'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:58.443807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a193081e23864453987cd63da6493049'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:58.478419 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-87e6523d-6b2c-451e-92bb-e2a791eb8644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a193081e23864453987cd63da6493049'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:58.489438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 831/1691] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:30:58 2022] DELETE /identity/v3/users/a193081e23864453987cd63da6493049 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:58.493796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:58.495350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:58.496632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:58.544929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:58.547584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:58.550873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:58.552705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:58.564969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 861/1692] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:30:58.802766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 832/1693] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:58.836118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:58.837624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:58.906408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:58.911692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:58.953857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9530eb-6a7b-4099-806e-4841417a36fa None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:58.983413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:58.985684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:58.986147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:58.986515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] PATH_INFO: `/v3/users/1e77bad300054ed58c520829b0721501` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:58.998874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e77bad300054ed58c520829b0721501)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:59.005138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:59.024430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:59.054598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99534e8-848d-4b7e-9dbc-947aa28daebf service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:59.093507 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:30:59.209479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e77bad300054ed58c520829b0721501'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:59.229648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e77bad300054ed58c520829b0721501'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:59.242954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:59.245062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:59.245694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e77bad300054ed58c520829b0721501'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:59.246365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e77bad300054ed58c520829b0721501'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:59.246715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:59.247214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:59.252578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:59.295922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:59.297408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c1d7b-09c1-46e9-afd3-521ecbe35113 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:59.319491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e77bad300054ed58c520829b0721501'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:30:59.329139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 862/1694] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:30:59.367172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 833/1695] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:30:58 2022] DELETE /identity/v3/users/1e77bad300054ed58c520829b0721501 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:30:59.368879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866dc3-f891-4a9d-be2c-e012b1e367ff None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:59.369171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866dc3-f891-4a9d-be2c-e012b1e367ff None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:59.369451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866dc3-f891-4a9d-be2c-e012b1e367ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:59.439796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-29866dc3-f891-4a9d-be2c-e012b1e367ff None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:30:59.451984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-29866dc3-f891-4a9d-be2c-e012b1e367ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:59.637280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 863/1696] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:30:59.670532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:30:59.678045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:30:59.911727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4df7b9-08ed-4f1f-acc3-b869e4e19e72 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:30:59.921800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:30:59.925449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:30:59.925449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:30:59.925672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:30:59.955985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:30:59.962127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:30:59.978747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 834/1697] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:00.142762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:00.143155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:00.143436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:00.213285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] MFA Rules not processed for user `75b75be070a147fba4788b0f51d8d0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:00.216025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:00.335840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 864/1698] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:00.351112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:00.353872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:00.517152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed807cae-fc36-4b41-a711-a575b8b2b088 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:00.575206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ca68a-e1be-4b13-a6fe-7442b11f1ef6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:00.576486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ca68a-e1be-4b13-a6fe-7442b11f1ef6 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:00.576774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ca68a-e1be-4b13-a6fe-7442b11f1ef6 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:00.579314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ca68a-e1be-4b13-a6fe-7442b11f1ef6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:00.605887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ca68a-e1be-4b13-a6fe-7442b11f1ef6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:00.608988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ca68a-e1be-4b13-a6fe-7442b11f1ef6 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:00.630235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 835/1699] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:00.921792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:00.934071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:00.961319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f334e-7d34-462f-b8db-540eab0231e8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:00.975257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:00.976463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:00.983993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:00.984297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] PATH_INFO: `/v3/projects/450cacbbb551460198179c4a0c7c18ac` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:00.986157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450cacbbb551460198179c4a0c7c18ac)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:00.988388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:01.108005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:01.120532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450cacbbb551460198179c4a0c7c18ac', 'request_id': 'req-31c66558-15ca-449d-9f46-6022bd260811'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.138392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450cacbbb551460198179c4a0c7c18ac', 'request_id': 'req-31c66558-15ca-449d-9f46-6022bd260811'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.146613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450cacbbb551460198179c4a0c7c18ac', 'request_id': 'req-31c66558-15ca-449d-9f46-6022bd260811'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.182257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] The token cache is being invalidate because project 450cacbbb551460198179c4a0c7c18ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:01.182557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.183968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.191304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 865/1700] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:00 2022] DELETE /identity/v3/projects/450cacbbb551460198179c4a0c7c18ac => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:01.193813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7a459-b645-4cfc-a50e-27201b443a75 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:01.194309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7a459-b645-4cfc-a50e-27201b443a75 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:01.194563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7a459-b645-4cfc-a50e-27201b443a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:01.243458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d4e7a459-b645-4cfc-a50e-27201b443a75 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:01.244969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d4e7a459-b645-4cfc-a50e-27201b443a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:01.270807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:01.274942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:01.276885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:01.423662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 836/1701] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:01.426802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:01.427858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:01.428374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:01.481406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:01.485397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:01.506494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 837/1702] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:01.511080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:01.520238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66558-15ca-449d-9f46-6022bd260811 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:01.538384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:01.539429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:01.539696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:01.539980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] PATH_INFO: `/v3/users/4c9812784d464aaea4e7862f173b5f4c` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:01.540847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084497e-bc4b-4364-84d6-206fc97dbd53 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:01.549528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9812784d464aaea4e7862f173b5f4c)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:01.551437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:01.552062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:01.552344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:01.552813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:01.553165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:01.554833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:01.555520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:01.558773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:01.585760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 838/1703] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:01.590092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:01.613123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce578fba-c278-4cf4-9531-503f9dd24128 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:01.625758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:01.626541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:01.630338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:01.631020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:01.635022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:01.639527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:01.670987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9812784d464aaea4e7862f173b5f4c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.693362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9812784d464aaea4e7862f173b5f4c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.694883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9812784d464aaea4e7862f173b5f4c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.712290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9812784d464aaea4e7862f173b5f4c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.735992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 839/1704] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:01 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:01.737546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:01.753092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:01.786919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c8b24264-4f9d-42ab-9d83-4b9fd737f530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9812784d464aaea4e7862f173b5f4c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:01.817245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 866/1705] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:01 2022] DELETE /identity/v3/users/4c9812784d464aaea4e7862f173b5f4c => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:01.825914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:01.825914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:01.827915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:01.902993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:01.904517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:02.013451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ffb55-3575-48fb-891c-c4a40dcdec8b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:02.059404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:02.096471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 867/1706] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:02.190181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:02.191795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:02.260527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:02.263633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.264817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.265714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.266550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:02.270546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:02.319860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 840/1707] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:01 2022] POST /identity/v3/users => generated 283 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:02.323726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:02.346271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cc0f0-ef18-471b-97bf-2ef8d1f1a035 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:02.351742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:02.352517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.352948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.353548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.354287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:02.356080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:02.359075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:02.377822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 841/1708] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:02.378912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c56ff-53e6-4153-a9c8-699ac3ad9c55 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:02.379420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:02.392710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:02.394808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.395328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.396042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.398753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565c794-4594-4f23-b52c-41b076847ff9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:02.411581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736aa29-2517-4ef6-8d66-20aea39de775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:02.412363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736aa29-2517-4ef6-8d66-20aea39de775 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.412772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736aa29-2517-4ef6-8d66-20aea39de775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.416018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736aa29-2517-4ef6-8d66-20aea39de775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.417247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736aa29-2517-4ef6-8d66-20aea39de775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:02.418455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:02.422268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:02.422960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2736aa29-2517-4ef6-8d66-20aea39de775 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:02.451074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 842/1710] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:02.451286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 868/1710] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:02.451384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.451966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:02.452361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.453262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.474436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3b1c7-9a8f-4323-9d45-2b33a45afa7d service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:02.487181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa775-c5f0-47b0-993e-65351fd4a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:02.488015 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa775-c5f0-47b0-993e-65351fd4a300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.489755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa775-c5f0-47b0-993e-65351fd4a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.490088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa775-c5f0-47b0-993e-65351fd4a300 admin admin] PATH_INFO: `/v3/projects/7e72cf46cf164c06a541eb9afd0714c9/users/9ded2bd1a8f845c9b9a3d3744d5b5779/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.498224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] MFA Rules not processed for user `84d2f3c3c5fb4cceabf9a35f64a1551c`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:02.503861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:02.563023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa775-c5f0-47b0-993e-65351fd4a300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e72cf46cf164c06a541eb9afd0714c9, user_id=9ded2bd1a8f845c9b9a3d3744d5b5779, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:02.573588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa775-c5f0-47b0-993e-65351fd4a300 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:02.627622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 869/1711] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:02 2022] PUT /identity/v3/projects/7e72cf46cf164c06a541eb9afd0714c9/users/9ded2bd1a8f845c9b9a3d3744d5b5779/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:02.642223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:02.643060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:02.644814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:02.765813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] MFA Rules not processed for user `9ded2bd1a8f845c9b9a3d3744d5b5779`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:02.766456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 843/1712] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:02.768003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:02.959127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 870/1713] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:02.972899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:02.974385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.012413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:03.015735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.115032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a155d-2f93-4b4c-8691-ff7178ab5e8a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:03.134758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:03.135558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:03.135849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:03.136124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] PATH_INFO: `/v3/projects/76b492740950465e92d7be4d3cbb9024` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:03.137703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b492740950465e92d7be4d3cbb9024)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:03.141744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:03.144661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:03.182670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc531fb-ccf5-4309-8594-06911156902c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:03.274560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b492740950465e92d7be4d3cbb9024', 'request_id': 'req-9b06740e-baa5-4c52-b790-8211401eaaed'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:03.282126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b492740950465e92d7be4d3cbb9024', 'request_id': 'req-9b06740e-baa5-4c52-b790-8211401eaaed'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:03.300180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b492740950465e92d7be4d3cbb9024', 'request_id': 'req-9b06740e-baa5-4c52-b790-8211401eaaed'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:03.343544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] The token cache is being invalidate because project 76b492740950465e92d7be4d3cbb9024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:03.343885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:03.345496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9b06740e-baa5-4c52-b790-8211401eaaed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:03.347060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:03.348275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:03.348400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 871/1714] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:03 2022] DELETE /identity/v3/projects/76b492740950465e92d7be4d3cbb9024 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:03.348855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:03.349273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:03.349942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:03.351039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:03.351356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:03.352846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.369437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:03.371691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e079149-b3f7-4fdc-a908-b18cab6cd0a3 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:03.391398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 844/1715] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:03.393201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd990b5c-eb22-4342-b799-7c05bf8bcf18 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:03.393507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd990b5c-eb22-4342-b799-7c05bf8bcf18 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:03.393783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd990b5c-eb22-4342-b799-7c05bf8bcf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:03.445174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cd990b5c-eb22-4342-b799-7c05bf8bcf18 None None] MFA Rules not processed for user `48a91f6797cc4604ba2606e0549234d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:03.446711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cd990b5c-eb22-4342-b799-7c05bf8bcf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.447869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] MFA Rules not processed for user `33c71ba6b2ad438d8593fa19eb617a34`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:03.449192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.554913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:03.660827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 872/1716] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:03.660827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:03.662372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.672687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 845/1717] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:03.674054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:03.674372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:03.675829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:03.734071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] MFA Rules not processed for user `33c71ba6b2ad438d8593fa19eb617a34`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:03.736329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.757643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 846/1718] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:03.758846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:03.762892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:03.852566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da114b-fb23-4937-af9e-171f35e05e83 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:03.861027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:03.861027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:03.861027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:03.861027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] PATH_INFO: `/v3/users/013ce7f1b61944939f75f3506e4eab7b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:03.862280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=013ce7f1b61944939f75f3506e4eab7b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:03.864389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:03.917581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:03.989879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013ce7f1b61944939f75f3506e4eab7b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.002068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013ce7f1b61944939f75f3506e4eab7b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.003220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013ce7f1b61944939f75f3506e4eab7b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.013121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e963e1-a881-4ca8-a9f8-7165d1dd2b0b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:04.023905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013ce7f1b61944939f75f3506e4eab7b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.029535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:04.030336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:04.030618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:04.030895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:04.038481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:04.060614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013ce7f1b61944939f75f3506e4eab7b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.070443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:04.075492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:04.082222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 873/1719] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:03 2022] DELETE /identity/v3/users/013ce7f1b61944939f75f3506e4eab7b => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:04.083280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:04.111252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:04.114951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 847/1720] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:04.229349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:04.230894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:04.258852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5760579-d470-4c7e-9c00-c6b0166ec3d8 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:04.285525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0e051-1008-4889-b549-efeb0f0087e0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:04.293233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:04.296474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:04.296740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:04.297062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] PATH_INFO: `/v3/projects/f542c80a8d1e4e86b2e3cfe2dc1ead8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:04.306335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f542c80a8d1e4e86b2e3cfe2dc1ead8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:04.306335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:04.407821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f542c80a8d1e4e86b2e3cfe2dc1ead8b', 'request_id': 'req-2e0d7b61-02e7-4fde-8767-87c4beaf3981'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.416334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:04.421693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:04.422075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:04.422768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] PATH_INFO: `/v3/users/e456d6b0e7b9430783ff5c0a442b82c2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:04.425746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f542c80a8d1e4e86b2e3cfe2dc1ead8b', 'request_id': 'req-2e0d7b61-02e7-4fde-8767-87c4beaf3981'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.426320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e456d6b0e7b9430783ff5c0a442b82c2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:04.433542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:04.446215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f542c80a8d1e4e86b2e3cfe2dc1ead8b', 'request_id': 'req-2e0d7b61-02e7-4fde-8767-87c4beaf3981'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.486962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] The token cache is being invalidate because project f542c80a8d1e4e86b2e3cfe2dc1ead8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:04.487350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.488494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.495192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 874/1721] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:04 2022] DELETE /identity/v3/projects/f542c80a8d1e4e86b2e3cfe2dc1ead8b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:04.497119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:04.502255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:04.564329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:04.570343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e456d6b0e7b9430783ff5c0a442b82c2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.586599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e456d6b0e7b9430783ff5c0a442b82c2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.592579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:04.604732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e456d6b0e7b9430783ff5c0a442b82c2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.605571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e456d6b0e7b9430783ff5c0a442b82c2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.627385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-baa58e31-f004-445b-a59c-a2aa797fbc98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e456d6b0e7b9430783ff5c0a442b82c2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.637770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 848/1722] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:04 2022] DELETE /identity/v3/users/e456d6b0e7b9430783ff5c0a442b82c2 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:04.642138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737712d-226b-430d-8190-5895b4996b51 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:04.642580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737712d-226b-430d-8190-5895b4996b51 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:04.642867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737712d-226b-430d-8190-5895b4996b51 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:04.649291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0d7b61-02e7-4fde-8767-87c4beaf3981 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:04.651543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 849/1723] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 14:31:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:31:04.652943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737712d-226b-430d-8190-5895b4996b51 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:04.655635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2737712d-226b-430d-8190-5895b4996b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:04.775809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737712d-226b-430d-8190-5895b4996b51 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:04.785490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:04.789837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:04.790049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:04.790338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] PATH_INFO: `/v3/users/a1d38c20ccd649ecbce25c62d4b2758a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:04.796526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1d38c20ccd649ecbce25c62d4b2758a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:04.798329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:04.917706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:04.918459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:04.922349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:04.922715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:04.934150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:04.957574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1d38c20ccd649ecbce25c62d4b2758a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:04.995711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1d38c20ccd649ecbce25c62d4b2758a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.020441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1d38c20ccd649ecbce25c62d4b2758a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.021949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1d38c20ccd649ecbce25c62d4b2758a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.059730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a980d04-1cf3-48fc-8397-c1df5b11dac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1d38c20ccd649ecbce25c62d4b2758a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.076287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 850/1724] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:04 2022] DELETE /identity/v3/users/a1d38c20ccd649ecbce25c62d4b2758a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:05.079279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.079279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.079279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.122029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:05.125990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.180192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:05.187989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab28fc-4558-4f00-93dc-c2a0239e0199 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:05.217638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 875/1725] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:05.222038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.223002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.223554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.288753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:05.337598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 851/1726] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:05.357975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:05.369426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.416311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8c41-287b-4a28-a017-d106081bfd49 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:05.417163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:05.421146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.447125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:05.447125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.447125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.447125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] PATH_INFO: `/v3/projects/3cb75c9e05684f5b90d7f4623397f9e4` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.447974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb75c9e05684f5b90d7f4623397f9e4)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:05.447974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:05.525888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 876/1727] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:05.527568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:05.530322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.541728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:05.546966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb75c9e05684f5b90d7f4623397f9e4', 'request_id': 'req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.552006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbd1b5-838f-4b2f-8b95-0712e9dadd5e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:05.559107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb75c9e05684f5b90d7f4623397f9e4', 'request_id': 'req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.578356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb75c9e05684f5b90d7f4623397f9e4', 'request_id': 'req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.596342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] The token cache is being invalidate because project 3cb75c9e05684f5b90d7f4623397f9e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:05.596660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.599568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-349121ed-688a-481e-ac4f-9d6bbc3cfa2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.605022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 852/1728] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:05 2022] DELETE /identity/v3/projects/3cb75c9e05684f5b90d7f4623397f9e4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:05.606631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.606920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.607228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.650863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:05.652487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.655858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:05.657184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.658988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.660254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.663342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.761156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:05.763322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:05.779749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 877/1729] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:05.781235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:05.782986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.797661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0036f81-7ea1-408f-91d0-f316e7c88343 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:05.804871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:05.805140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 853/1730] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:05.805741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.806030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.806345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.812726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:05.814122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.815699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.826924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2e5e-dc3f-4916-97d1-fff072dfea9c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:05.832475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:05.833124 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.833401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.833674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] PATH_INFO: `/v3/projects/2044ed14955242bfa45c10cd3e370ffd` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.834862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2044ed14955242bfa45c10cd3e370ffd)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:05.836839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:05.846038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:05.850636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bbfb6-b1d2-453b-bbf1-7cb3bd9f67b5 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:05.884978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 878/1731] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:05.887361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.887749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.888298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.928021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:05.930709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:05.932149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2044ed14955242bfa45c10cd3e370ffd', 'request_id': 'req-9a82d14d-b949-443b-8134-e888def77df7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.945485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2044ed14955242bfa45c10cd3e370ffd', 'request_id': 'req-9a82d14d-b949-443b-8134-e888def77df7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.952280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2044ed14955242bfa45c10cd3e370ffd', 'request_id': 'req-9a82d14d-b949-443b-8134-e888def77df7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.974275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] The token cache is being invalidate because project 2044ed14955242bfa45c10cd3e370ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:05.974688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.975795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9a82d14d-b949-443b-8134-e888def77df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:05.981177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 854/1732] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:05 2022] DELETE /identity/v3/projects/2044ed14955242bfa45c10cd3e370ffd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:05.983189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3a20-e7af-44a3-97ba-40f32c492e07 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:05.983612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3a20-e7af-44a3-97ba-40f32c492e07 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:05.983893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3a20-e7af-44a3-97ba-40f32c492e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:05.996605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:06.012121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3c1d3a20-e7af-44a3-97ba-40f32c492e07 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:06.014256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3c1d3a20-e7af-44a3-97ba-40f32c492e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:06.121167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 879/1733] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:06.125784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.156410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 855/1734] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:06.156634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2df9e-17aa-45ef-9695-7fff993c15cf None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.158842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.159268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.160434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.169602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.170496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.170783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.172028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.173256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:06.173577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.177716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.202862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:06.203811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:06.210549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 880/1735] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:06.215054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.222385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 856/1736] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:06.230057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.239452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09937d36-8bb6-4163-a4ce-5f283e46c2d2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.254933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.255569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.255855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.256132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.256825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.258822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.274501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2da468-10d2-45ad-8eac-904a993c0056 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.284650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.285412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.285876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.286152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.288455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:06.288794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.291676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.309372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 881/1737] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/projects => generated 355 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:06.313828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.321314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 857/1738] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:06.324783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:06.325922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.337649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f68e2-9980-43d0-9474-b6ab849eef80 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.431660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760dd5f-1199-4d77-bdbf-08425252f85e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.441763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32123fb-cc13-483e-a8e6-c1f8e84bbcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.442498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32123fb-cc13-483e-a8e6-c1f8e84bbcd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.442787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32123fb-cc13-483e-a8e6-c1f8e84bbcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.443177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32123fb-cc13-483e-a8e6-c1f8e84bbcd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.443832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32123fb-cc13-483e-a8e6-c1f8e84bbcd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.445864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32123fb-cc13-483e-a8e6-c1f8e84bbcd9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.464767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd92b0-c358-49bf-9b92-edf8ec5d2ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.465484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd92b0-c358-49bf-9b92-edf8ec5d2ede admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.465735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd92b0-c358-49bf-9b92-edf8ec5d2ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.466445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd92b0-c358-49bf-9b92-edf8ec5d2ede admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.466688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd92b0-c358-49bf-9b92-edf8ec5d2ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.468552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd92b0-c358-49bf-9b92-edf8ec5d2ede admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.492437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 882/1739] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/users => generated 278 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:06.492437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05655-ca9e-4571-b36c-07f0d6251fd3 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.492437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05655-ca9e-4571-b36c-07f0d6251fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.495608 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05655-ca9e-4571-b36c-07f0d6251fd3 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.498782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 883/1740] 10.222.0.50 () {54 vars in 912 bytes} [Mon Aug 8 14:31:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:31:06.500452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05655-ca9e-4571-b36c-07f0d6251fd3 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.527826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05655-ca9e-4571-b36c-07f0d6251fd3 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.542333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.547047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.547047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.547047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.547047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.549326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.557112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 858/1741] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:06.561564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.561856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.562130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.571412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 884/1742] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:06.572391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.596225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3c869-aa4e-40a0-b31e-d2457896c94b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.626901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:06.628176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:06.719022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.719746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.720031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.720309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.720947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.722880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 859/1743] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:06.723578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.726884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.737462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688a1a9-6afb-4ee7-9ceb-8235bd15a8de None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.743736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.744675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.744973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.745253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.746049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.748014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.765060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 860/1744] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:06.765060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.765348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:06.771967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 885/1745] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:06.771967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.781494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24793b-5b9d-4d16-afda-d502a0341f8d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.788563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.789290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.789583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.789865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] PATH_INFO: `/v3/projects/d8ab4c0542ff46878576d61c3ca8c9fc` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.791488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696d3ae-98fd-45e4-9d90-c349da9c08bd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:06.791616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ab4c0542ff46878576d61c3ca8c9fc)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.793888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.798303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:06.800520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:06.801591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.802103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.802607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] PATH_INFO: `/v3/projects/822de5994c174b92bb31327f1ed2dd00/users/00b5b80b8d0140bb86ac003362f7e035/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.822763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] RBAC: Authorizing `identity:create_grant(project_id=822de5994c174b92bb31327f1ed2dd00, user_id=00b5b80b8d0140bb86ac003362f7e035, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:06.826886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:06.848538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 886/1746] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:06 2022] PUT /identity/v3/projects/822de5994c174b92bb31327f1ed2dd00/users/00b5b80b8d0140bb86ac003362f7e035/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:06.850260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:06.911224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ab4c0542ff46878576d61c3ca8c9fc', 'request_id': 'req-4015e040-9ff5-4c8d-8964-5cce7940d959'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:06.922310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ab4c0542ff46878576d61c3ca8c9fc', 'request_id': 'req-4015e040-9ff5-4c8d-8964-5cce7940d959'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:06.930540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ab4c0542ff46878576d61c3ca8c9fc', 'request_id': 'req-4015e040-9ff5-4c8d-8964-5cce7940d959'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:06.953529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] The token cache is being invalidate because project d8ab4c0542ff46878576d61c3ca8c9fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:06.954349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:06.955882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4015e040-9ff5-4c8d-8964-5cce7940d959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:06.959372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 861/1747] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:06 2022] DELETE /identity/v3/projects/d8ab4c0542ff46878576d61c3ca8c9fc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:06.963263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:06.963920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:06.965474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:06.968173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:07.007615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139223e-6ba5-4b50-b69d-7c6ce710628e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:07.044502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] MFA Rules not processed for user `00b5b80b8d0140bb86ac003362f7e035`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:07.047918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.172033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:07.175692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.175930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.176436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.177068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:07.179957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:07.213087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 862/1748] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:07.214725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:07.218338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.232701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 887/1749] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:07.235001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:07.242693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.280343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1dd8dd-0c9e-446b-bfce-ce62ac0743c1 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:07.288574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:07.289236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.289518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.289793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] PATH_INFO: `/v3/users/65099867e2fd4cdc8c37421078bd57eb` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.296228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65099867e2fd4cdc8c37421078bd57eb)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:07.298234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:07.326926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53001c91-ad56-4bcc-bee7-3420d0170744 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:07.346409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:07.347198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.347580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.347930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.352418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.372107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:07.373674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6accb7-3732-4652-9a8f-8f646ff89a1d service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:07.384634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 863/1750] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:07.386046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.386411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.386764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.405299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65099867e2fd4cdc8c37421078bd57eb'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:07.411927 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] MFA Rules not processed for user `75b75be070a147fba4788b0f51d8d0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:07.414425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.426583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65099867e2fd4cdc8c37421078bd57eb'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:07.427511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65099867e2fd4cdc8c37421078bd57eb'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:07.442264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65099867e2fd4cdc8c37421078bd57eb'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:07.474810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:07.483899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65099867e2fd4cdc8c37421078bd57eb'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:07.494350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 888/1751] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:07 2022] DELETE /identity/v3/users/65099867e2fd4cdc8c37421078bd57eb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:07.498980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:07.501847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.575880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 864/1752] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:07.577880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:07.582722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:07.608086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af5d99-ea5a-43c7-8b12-0a79a2715f30 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:07.753489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ed709-88a8-4615-8a01-4e2203ae96d7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:07.767288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25822a02-907c-4deb-ac8f-cf51e34c359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:07.767951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25822a02-907c-4deb-ac8f-cf51e34c359b admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.768229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25822a02-907c-4deb-ac8f-cf51e34c359b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.770349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25822a02-907c-4deb-ac8f-cf51e34c359b admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.770349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25822a02-907c-4deb-ac8f-cf51e34c359b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:07.772795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25822a02-907c-4deb-ac8f-cf51e34c359b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:07.801611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:07.820547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dcc0ba-e654-4830-a5ee-a115f4736656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:07.821378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc0ba-e654-4830-a5ee-a115f4736656 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.821876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc0ba-e654-4830-a5ee-a115f4736656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.822321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dcc0ba-e654-4830-a5ee-a115f4736656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.823160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcc0ba-e654-4830-a5ee-a115f4736656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:07.825986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dcc0ba-e654-4830-a5ee-a115f4736656 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:07.839724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 865/1753] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:07.841283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abfa26-9b2b-4637-be2a-021075ef2a1c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.841889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abfa26-9b2b-4637-be2a-021075ef2a1c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.842552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abfa26-9b2b-4637-be2a-021075ef2a1c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.844580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 866/1754] 10.222.0.50 () {54 vars in 961 bytes} [Mon Aug 8 14:31:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:31:07.845525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abfa26-9b2b-4637-be2a-021075ef2a1c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:07.864854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 889/1755] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:07 2022] POST /identity/v3/projects => generated 355 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:07.867194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:07.867479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:07.867759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:07.872239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abfa26-9b2b-4637-be2a-021075ef2a1c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:07.903632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:07.906978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.012449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.013138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.013401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.013673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] PATH_INFO: `/v3/projects/0fb58a0ad64943ccb3dd65996e541905/users/12d239d71a30490ea58f06ed06b949b7/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.054048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb58a0ad64943ccb3dd65996e541905, user_id=12d239d71a30490ea58f06ed06b949b7, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:08.061902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:08.069265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 890/1756] 10.222.0.50 () {58 vars in 1058 bytes} [Mon Aug 8 14:31:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:08.077536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:08.097808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 867/1757] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:07 2022] PUT /identity/v3/projects/0fb58a0ad64943ccb3dd65996e541905/users/12d239d71a30490ea58f06ed06b949b7/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:08.101146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:08.117736 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019004a-9fb8-4d0f-9089-af34c5185b45 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:08.273498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:08.302463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa5317-8ffc-4f5a-893f-e3a65aa9d5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.303824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa5317-8ffc-4f5a-893f-e3a65aa9d5ae admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.304112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa5317-8ffc-4f5a-893f-e3a65aa9d5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.304514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa5317-8ffc-4f5a-893f-e3a65aa9d5ae admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.315117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa5317-8ffc-4f5a-893f-e3a65aa9d5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:08.315117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa5317-8ffc-4f5a-893f-e3a65aa9d5ae admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:08.348310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2f59f-369f-4a95-bcd9-fa7d42f1a285 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:08.360419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 891/1758] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:08 2022] POST /identity/v3/users => generated 284 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:08.362407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.362962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.363518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.370060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.373112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.374780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.375940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.384829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.429844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:08.431706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:08.442400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] MFA Rules not processed for user `12d239d71a30490ea58f06ed06b949b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:08.443698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.468828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 868/1759] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:08.470969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:08.475773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.527853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccca097-90f1-42cd-b27a-2db2699cbd08 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:08.541627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.542740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.543082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.543442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] PATH_INFO: `/v3/projects/b311c0cd195148d48637bfd66661ed17` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.544898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b311c0cd195148d48637bfd66661ed17)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:08.547660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:08.602159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 892/1760] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:08.605554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:08.624554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:08.638413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0063d9b-45c3-4bf7-97de-f48826dc2cf9 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:08.663899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b311c0cd195148d48637bfd66661ed17', 'request_id': 'req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:08.678623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b311c0cd195148d48637bfd66661ed17', 'request_id': 'req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:08.694770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b311c0cd195148d48637bfd66661ed17', 'request_id': 'req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:08.753664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe0b14f-70a8-4450-88c0-2bf25fd54503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.754389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b14f-70a8-4450-88c0-2bf25fd54503 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.756060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b14f-70a8-4450-88c0-2bf25fd54503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.756498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe0b14f-70a8-4450-88c0-2bf25fd54503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.759510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0b14f-70a8-4450-88c0-2bf25fd54503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:08.761284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe0b14f-70a8-4450-88c0-2bf25fd54503 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:08.791579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] The token cache is being invalidate because project b311c0cd195148d48637bfd66661ed17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:08.793384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:08.796142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:08.804451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 869/1761] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:08 2022] DELETE /identity/v3/projects/b311c0cd195148d48637bfd66661ed17 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:08.807496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:08.811508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.823925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 893/1762] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:08.825527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.825829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.826209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.859823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d421ee-b88d-40e0-952c-b6a2cbb0dd99 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:08.885563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6394506-0952-4f91-9395-9d454835cbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.886240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6394506-0952-4f91-9395-9d454835cbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.887036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6394506-0952-4f91-9395-9d454835cbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.887353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6394506-0952-4f91-9395-9d454835cbc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:08.887993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6394506-0952-4f91-9395-9d454835cbc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:08.889404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:08.894212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6394506-0952-4f91-9395-9d454835cbc6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:08.894570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.921958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 894/1763] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:08.925244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:08.927932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:08.963417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f1ed6-6fd9-48e7-bdcd-ec276cfeb54a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:08.972114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:08.974254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:08.974538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:08.974817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] PATH_INFO: `/v3/projects/7627faaaad42449daf5f45ab212ed7d1/users/2a8365b8a71f456a8167712dc565fe07/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.015181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 870/1764] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:08 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:09.015181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.015181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.015181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.061786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7627faaaad42449daf5f45ab212ed7d1, user_id=2a8365b8a71f456a8167712dc565fe07, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.070655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.092707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:09.100263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:09.149521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 895/1765] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:08 2022] PUT /identity/v3/projects/7627faaaad42449daf5f45ab212ed7d1/users/2a8365b8a71f456a8167712dc565fe07/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:09.154724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.344289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee91704-5c2e-4966-8366-cc4cb0845b1a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:09.355474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 871/1766] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:09.358266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.362020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4546-9d5a-4d6a-80e5-eed00889cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:09.362744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4546-9d5a-4d6a-80e5-eed00889cd22 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.363117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4546-9d5a-4d6a-80e5-eed00889cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.363514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4546-9d5a-4d6a-80e5-eed00889cd22 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.364322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e4546-9d5a-4d6a-80e5-eed00889cd22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.384772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e4546-9d5a-4d6a-80e5-eed00889cd22 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.400918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ff65d-aaef-416f-a785-a5149c24546e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:09.423433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:09.424101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.424383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.424657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.425379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.426914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.449855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 896/1767] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:09 2022] POST /identity/v3/users => generated 300 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:09.454334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.454626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.454901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.473286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 872/1768] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:09.479863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.504050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] MFA Rules not processed for user `48a91f6797cc4604ba2606e0549234d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:09.508128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:09.518748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d429c85-925f-4bbf-b851-a7a6ef101c08 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:09.563792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:09.564429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.564713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.565362 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.565658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:09.566032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.580239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.629110 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 873/1769] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:09.635173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.676596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd327a7-abfd-48da-8f7a-07ba415164db admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:09.692450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:09.695779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.696115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.696528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.698249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.702540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.730822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 874/1770] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:09.734026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.740869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 897/1771] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:09.742516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.752277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:09.761812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:09.770677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dc229-25c2-42f2-93e5-4ab6295f3c7a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:09.773771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0d3cc-48d1-4fb2-bec1-c541c0dd431c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:09.789114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435c79e-4371-44da-a939-3a7b580e228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:09.789695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:09.789851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c79e-4371-44da-a939-3a7b580e228c admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.790017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c79e-4371-44da-a939-3a7b580e228c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.790326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435c79e-4371-44da-a939-3a7b580e228c admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.790432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.790657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.790911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435c79e-4371-44da-a939-3a7b580e228c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.791029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] PATH_INFO: `/v3/projects/7627faaaad42449daf5f45ab212ed7d1/users/2a8365b8a71f456a8167712dc565fe07/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:09.801065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7627faaaad42449daf5f45ab212ed7d1, user_id=2a8365b8a71f456a8167712dc565fe07, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:09.807133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.810879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435c79e-4371-44da-a939-3a7b580e228c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:09.862373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 875/1772] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:09 2022] PUT /identity/v3/projects/7627faaaad42449daf5f45ab212ed7d1/users/2a8365b8a71f456a8167712dc565fe07/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:09.876579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:09.929088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 898/1773] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:09 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:09.937917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:09.938202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:09.938478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.052737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] MFA Rules not processed for user `2a8365b8a71f456a8167712dc565fe07`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:10.055131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:10.108572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee668113-01d2-4d05-9df9-eda12082f3ae admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.348137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db045-daca-41de-9077-fe89eda250de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.348921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db045-daca-41de-9077-fe89eda250de admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.349208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db045-daca-41de-9077-fe89eda250de admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.349492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325db045-daca-41de-9077-fe89eda250de admin admin] PATH_INFO: `/v3/projects/a42a3b4d6da44df7b77720aee25f809b/users/90e25922bda64d37b01dcd03b5996192/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.400339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 899/1774] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:10.405474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:10.446776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325db045-daca-41de-9077-fe89eda250de admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42a3b4d6da44df7b77720aee25f809b, user_id=90e25922bda64d37b01dcd03b5996192, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.452912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325db045-daca-41de-9077-fe89eda250de admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.456344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37918-eeb5-4e11-b3d9-2493845eab56 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.484409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.491702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.492004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.492284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.492971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.505155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 876/1775] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:09 2022] PUT /identity/v3/projects/a42a3b4d6da44df7b77720aee25f809b/users/90e25922bda64d37b01dcd03b5996192/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:10.509045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.509331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db045-daca-41de-9077-fe89eda250de admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:10.516322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-325db045-daca-41de-9077-fe89eda250de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:10.614853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 900/1776] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:10 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:10.618359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:10.658703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8ef39-58c9-4cd5-ba1c-9813cb644685 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.743630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325db045-daca-41de-9077-fe89eda250de admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.757013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.758191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.759462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.760259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.793020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.794213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.794568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.794898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.795916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.798393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.820692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 901/1777] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:10.830432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:10.835469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:10.850617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328027c-5b08-4ff3-8e61-b1afdef0fcb4 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.871752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.879374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.879784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.880073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.880862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.883200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.909939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.910113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 902/1778] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:10.911031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:10.913510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.934838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e48950-7a8c-4cc3-9745-12f6cc70ef90 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.940952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 877/1779] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:10.942060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:10.947239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8e8fa5-8c27-489d-a55a-b9daed01d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.947754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8fa5-8c27-489d-a55a-b9daed01d40f admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.948012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8fa5-8c27-489d-a55a-b9daed01d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.948291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8e8fa5-8c27-489d-a55a-b9daed01d40f admin admin] PATH_INFO: `/v3/projects/a42a3b4d6da44df7b77720aee25f809b/users/90e25922bda64d37b01dcd03b5996192/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.957123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e8fa5-8c27-489d-a55a-b9daed01d40f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42a3b4d6da44df7b77720aee25f809b, user_id=90e25922bda64d37b01dcd03b5996192, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.960566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8e8fa5-8c27-489d-a55a-b9daed01d40f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.966031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142db75-059b-4b8f-8749-1fd16a94cfee service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:10.975875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:10.977691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.977975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.978247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:10.978900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:10.984357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:10.994241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 903/1780] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:10 2022] PUT /identity/v3/projects/a42a3b4d6da44df7b77720aee25f809b/users/90e25922bda64d37b01dcd03b5996192/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:10.996124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:10.996586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:10.996733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:11.005222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 878/1781] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:11.008638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:11.030952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e80b-5214-4f83-b763-bc2323998981 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:11.059676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] MFA Rules not processed for user `90e25922bda64d37b01dcd03b5996192`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:11.064733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:11.092854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:11.201353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:11.204524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:11.208047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:11.208420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] PATH_INFO: `/v3/projects/23c1adbfe1ff40f09ec058bfd706de9a/users/5f49d3d42cd44838a7fd12d94a4f089a/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:11.238714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 904/1782] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:11.239807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:11.255674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c1adbfe1ff40f09ec058bfd706de9a, user_id=5f49d3d42cd44838a7fd12d94a4f089a, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:11.265955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:11.325055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 879/1783] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:11 2022] PUT /identity/v3/projects/23c1adbfe1ff40f09ec058bfd706de9a/users/5f49d3d42cd44838a7fd12d94a4f089a/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:11.325055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:11.325055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:11.385473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a32d15-3118-4cc1-9dec-c01acb3f0617 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:11.458859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c2b34-5ac1-43f4-86da-650cfa6282f7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:11.474301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55ecd9-aaae-431f-b86b-7f7ff6b65ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:11.474972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55ecd9-aaae-431f-b86b-7f7ff6b65ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:11.475341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55ecd9-aaae-431f-b86b-7f7ff6b65ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:11.478993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55ecd9-aaae-431f-b86b-7f7ff6b65ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:11.534556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:11.535236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:11.535639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:11.536006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:11.538643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:11.571674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:11.576824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55ecd9-aaae-431f-b86b-7f7ff6b65ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:11.578732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55ecd9-aaae-431f-b86b-7f7ff6b65ef3 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:11.590609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 880/1784] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:11.592087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:11.592716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:11.593045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:11.626239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] MFA Rules not processed for user `5f49d3d42cd44838a7fd12d94a4f089a`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:11.628243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:11.672396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:11.676292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:11.714821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 905/1785] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:11.837986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 881/1786] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:11.841866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:11.853606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:11.890583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:12.043702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d39122-0dd4-4380-a23c-1178028a39c1 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:12.044531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669db6ba-7cd5-4f7a-86c2-f4dda661205b service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:12.056749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:12.057752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.058242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.058665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] PATH_INFO: `/v3/users/12d239d71a30490ea58f06ed06b949b7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.060239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d239d71a30490ea58f06ed06b949b7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:12.061649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:12.062464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.062746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.063289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.063829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:12.063932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:12.065997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:12.173972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 906/1787] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:11 2022] POST /identity/v3/projects => generated 379 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:12.175587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:12.199347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d239d71a30490ea58f06ed06b949b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:12.224408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d239d71a30490ea58f06ed06b949b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:12.239476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d239d71a30490ea58f06ed06b949b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:12.240215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d239d71a30490ea58f06ed06b949b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:12.265195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-513c59de-4309-4070-bc3c-693948fe8a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d239d71a30490ea58f06ed06b949b7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:12.278385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 882/1788] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:11 2022] DELETE /identity/v3/users/12d239d71a30490ea58f06ed06b949b7 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:12.282507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.282793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.283077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.317431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:12.384339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369be6cf-56f9-49bc-b502-99b642ed6780 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:12.400845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:12.402214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:12.592675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 883/1789] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:12.630613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:12.633072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:12.633526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.633956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.634313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.634552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:12.635993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:12.645264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:12.707834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 907/1790] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:12 2022] POST /identity/v3/users => generated 290 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:12.712374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:12.715275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:12.800967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e0319-124b-48b2-956b-ab70aca3732f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:12.817011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:12.817938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.818385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.819518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.820306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:12.826315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:12.852292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 908/1791] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:12.855409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:12.909428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaed0f-4803-4e2f-a62c-8ed0973100b6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:12.913754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efafe1-02b7-4905-bc90-558f1533d9ab admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:12.926925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:12.930858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.932480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.933140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fd4dc-5925-4247-9181-e85cf99afef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:12.934410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.935127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd4dc-5925-4247-9181-e85cf99afef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.939496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd4dc-5925-4247-9181-e85cf99afef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.940401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd4dc-5925-4247-9181-e85cf99afef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:12.942209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd4dc-5925-4247-9181-e85cf99afef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:12.948325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd4dc-5925-4247-9181-e85cf99afef5 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:12.983231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:12.983231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:12.989055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 909/1792] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:12.992094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:12.992392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:12.992716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:13.014078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 884/1793] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:13.016588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:13.053714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7eb93-e624-4d46-af4c-b0c8fd256a60 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:13.054549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:13.072236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55c472-d678-4fc6-aa2a-9328e4955133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:13.073315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55c472-d678-4fc6-aa2a-9328e4955133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:13.074978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55c472-d678-4fc6-aa2a-9328e4955133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:13.075884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55c472-d678-4fc6-aa2a-9328e4955133 admin admin] PATH_INFO: `/v3/projects/e8058f9450724038b3aa281b8c35a298/users/bccb1a36b282443d8e136d3c79b52948/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:13.136234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55c472-d678-4fc6-aa2a-9328e4955133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8058f9450724038b3aa281b8c35a298, user_id=bccb1a36b282443d8e136d3c79b52948, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:13.145012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55c472-d678-4fc6-aa2a-9328e4955133 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:13.154557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:13.156944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:13.185199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 885/1794] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:13 2022] PUT /identity/v3/projects/e8058f9450724038b3aa281b8c35a298/users/bccb1a36b282443d8e136d3c79b52948/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:13.194997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:13.195543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:13.196002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:13.241884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] MFA Rules not processed for user `bccb1a36b282443d8e136d3c79b52948`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:13.243219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:13.355855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 886/1795] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:13.363198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:13.400080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:13.447914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 910/1796] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:31:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:13.534019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeff914-52ba-448e-9848-de1e41c51ba2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:13.547976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:13.548684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:13.555382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:13.555916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:13.558420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:13.563827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:13.632655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 887/1797] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:13 2022] POST /identity/v3/projects => generated 379 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:13.638465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:13.674275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94af58c-3750-437f-bb93-51ac688d5272 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:13.694038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:13.696258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:13.930467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:13.931141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:13.931416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:13.931693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:13.946432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:13.950096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:14.000780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5c8ec-c11e-4930-a919-789be21dbead admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:14.020677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 911/1798] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:13 2022] POST /identity/v3/users => generated 296 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:14.020828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:14.020828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:14.021356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:14.022256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:14.022751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:14.052768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74c2bb-b082-45da-93a4-00a9c0120e83 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:14.063222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:14.063841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:14.064138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:14.068509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:14.069420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:14.074935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:14.119261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 912/1799] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:14.124628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:14.150471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3f47-ba16-4c4e-8e9a-e09f266e58e9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:14.159595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:14.160280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:14.160564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:14.161030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] PATH_INFO: `/v3/projects/6a5df6c9e73d41029590dabdd23d24dc/users/3b9108cf7aed4d54819fb77aa17429ec/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:14.180347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5df6c9e73d41029590dabdd23d24dc, user_id=3b9108cf7aed4d54819fb77aa17429ec, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:14.196125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:14.198158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:14.200045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:14.239573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 913/1800] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:14 2022] PUT /identity/v3/projects/6a5df6c9e73d41029590dabdd23d24dc/users/3b9108cf7aed4d54819fb77aa17429ec/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:14.243299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:14.243694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 888/1801] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:31:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 14:31:14.423459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:14.458846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027594b6-f702-4fd3-8cac-fd1b517f6070 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:14.466551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2067e9-9b3f-4fcd-bd43-8eae75342c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:14.467427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2067e9-9b3f-4fcd-bd43-8eae75342c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:14.467719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2067e9-9b3f-4fcd-bd43-8eae75342c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:14.468749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2067e9-9b3f-4fcd-bd43-8eae75342c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:14.468749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2067e9-9b3f-4fcd-bd43-8eae75342c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:14.470254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2067e9-9b3f-4fcd-bd43-8eae75342c0a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:14.487749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 914/1802] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:14.491750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:14.526144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6e1b9-8621-460a-a16f-0f59a293a87e service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:14.547770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:14.549283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:14.551611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:14.553111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] PATH_INFO: `/v3/projects/6a5df6c9e73d41029590dabdd23d24dc/users/3b9108cf7aed4d54819fb77aa17429ec/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:14.565546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5df6c9e73d41029590dabdd23d24dc, user_id=3b9108cf7aed4d54819fb77aa17429ec, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:14.575428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:14.625998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 889/1803] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:14 2022] PUT /identity/v3/projects/6a5df6c9e73d41029590dabdd23d24dc/users/3b9108cf7aed4d54819fb77aa17429ec/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:14.638072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:14.638641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:14.639574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:14.751948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] MFA Rules not processed for user `3b9108cf7aed4d54819fb77aa17429ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:14.759410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:15.000821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 915/1804] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:15.160148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:15.163463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:15.179411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:15.388554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b15d537-09b1-4354-b114-d4281021878e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:15.393103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:15.394005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:15.394475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:15.394924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:15.434875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:15.436473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:15.473372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 890/1805] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:15.492693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:15.494666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:15.776756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:15.781743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:15.833081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610e696-e8aa-4ed7-bb2b-9a30d71e725f service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:15.844520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:15.845134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:15.845649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:15.845953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:15.848639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:15.864030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b034e6eb-4b9f-442d-ac0a-3eb6929e190c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:15.890092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:15.890768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:15.891357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:15.891635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] PATH_INFO: `/v3/projects/0fb58a0ad64943ccb3dd65996e541905` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:15.895362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb58a0ad64943ccb3dd65996e541905)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:15.897476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:16.021199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb58a0ad64943ccb3dd65996e541905', 'request_id': 'req-667fed6c-a770-413d-808d-256c9ffe1d1b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.046662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb58a0ad64943ccb3dd65996e541905', 'request_id': 'req-667fed6c-a770-413d-808d-256c9ffe1d1b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.055772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb58a0ad64943ccb3dd65996e541905', 'request_id': 'req-667fed6c-a770-413d-808d-256c9ffe1d1b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.088806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:16.089995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:16.095694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:16.112490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] The token cache is being invalidate because project 0fb58a0ad64943ccb3dd65996e541905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:16.113716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 891/1806] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:16.114005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.115075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-667fed6c-a770-413d-808d-256c9ffe1d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.118005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 916/1807] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:15 2022] DELETE /identity/v3/projects/0fb58a0ad64943ccb3dd65996e541905 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:16.123309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:16.160405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267bd0f2-b20a-4a76-9f98-415f7fd64c5a service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:16.391459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:16.392207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:16.392486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:16.392764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] PATH_INFO: `/v3/users/90e25922bda64d37b01dcd03b5996192` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:16.397758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90e25922bda64d37b01dcd03b5996192)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:16.402507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:16.609245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90e25922bda64d37b01dcd03b5996192'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.610830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:16.632780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90e25922bda64d37b01dcd03b5996192'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.651195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90e25922bda64d37b01dcd03b5996192'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.652004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90e25922bda64d37b01dcd03b5996192'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.710529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90e25922bda64d37b01dcd03b5996192'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:16.731916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 892/1808] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:16 2022] DELETE /identity/v3/users/90e25922bda64d37b01dcd03b5996192 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:16.932297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:16.932907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:16.932907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:16.981736 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:16.988303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:17.254590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 917/1809] 10.222.0.16 () {58 vars in 1015 bytes} [Mon Aug 8 14:31:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:17.280844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:17.284678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:17.345109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa8883-2fc3-418d-abc1-ef7e8f482535 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:17.363838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e77c8-28da-46a6-9652-0598db2009b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:17.374962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e77c8-28da-46a6-9652-0598db2009b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:17.385793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e77c8-28da-46a6-9652-0598db2009b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:17.386164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e77c8-28da-46a6-9652-0598db2009b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:17.431531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e77c8-28da-46a6-9652-0598db2009b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:17.438846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e77c8-28da-46a6-9652-0598db2009b8 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:17.480491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 893/1810] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:19.007805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:19.008622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:19.232375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f78311-b093-441a-b6e2-3e70422d88d8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:19.251818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:19.254127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:19.255797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:19.257282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] PATH_INFO: `/v3/projects/a42a3b4d6da44df7b77720aee25f809b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:19.265204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42a3b4d6da44df7b77720aee25f809b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:19.279553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:19.402304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42a3b4d6da44df7b77720aee25f809b', 'request_id': 'req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:19.412587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42a3b4d6da44df7b77720aee25f809b', 'request_id': 'req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:19.424942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42a3b4d6da44df7b77720aee25f809b', 'request_id': 'req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:19.447251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] The token cache is being invalidate because project a42a3b4d6da44df7b77720aee25f809b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:19.447552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:19.448218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-38252fb6-41a4-4e3e-ab4f-3b2c9c897e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:19.450050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 918/1811] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:19 2022] DELETE /identity/v3/projects/a42a3b4d6da44df7b77720aee25f809b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:19.455671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:19.456198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:19.456198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:19.513926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:19.515375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:19.652515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 894/1812] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:19.664031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:19.664771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:19.665195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:19.720150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:19.741189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:19.780991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 919/1813] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:19.781959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:19.798675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdacb621-dc61-45a5-a926-d1f48c8aa6de None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:19.808474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:19.809587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:19.810472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:19.810916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:19.812376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:19.813025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:19.816253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:19.853774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 895/1814] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:31:19.859874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:19.902238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83371130-c9e1-40b5-9c45-a9ddd58d3069 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:19.927310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:19.928859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:19.931730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:19.932228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:19.933096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:19.937653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:19.938608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:19.940410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:19.962243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b4afe-ce0e-455f-9d2b-e1d28f580ebf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:19.973099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:19.974104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:19.974364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:19.977269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] PATH_INFO: `/v3/users/9ded2bd1a8f845c9b9a3d3744d5b5779` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:19.978832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ded2bd1a8f845c9b9a3d3744d5b5779)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:19.982211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:19.999487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 920/1815] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:19 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:20.000787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:20.012786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:20.104226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cdd19-e7fc-4485-8f97-e92bfa15bc73 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:20.113466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:20.114651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:20.115110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:20.116025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:20.117003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:20.120204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:20.172976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ded2bd1a8f845c9b9a3d3744d5b5779'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:20.196341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 921/1816] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:19 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:20.198283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:20.213627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ded2bd1a8f845c9b9a3d3744d5b5779'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:20.231491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102b24d-a1c6-4c5a-8538-535ae749b009 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:20.242333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ded2bd1a8f845c9b9a3d3744d5b5779'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:20.243339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ded2bd1a8f845c9b9a3d3744d5b5779'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:20.249264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:20.250506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:20.250799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:20.251727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:20.253119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:20.257077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:20.279765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 922/1817] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:20.282583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:20.285437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:20.300003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2c9a3af5-f839-42d8-941f-6f694d5a658a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ded2bd1a8f845c9b9a3d3744d5b5779'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:20.328184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4100fe-8e53-4947-ae7a-3fc2707a77c0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:20.331781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 896/1818] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:19 2022] DELETE /identity/v3/users/9ded2bd1a8f845c9b9a3d3744d5b5779 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:20.336626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3587c-2935-43cc-b791-f16c6c260168 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:20.336993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3587c-2935-43cc-b791-f16c6c260168 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:20.337189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3587c-2935-43cc-b791-f16c6c260168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:20.451699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-44d3587c-2935-43cc-b791-f16c6c260168 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:20.458420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-44d3587c-2935-43cc-b791-f16c6c260168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:20.470914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:20.509002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:20.509661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:20.509944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:20.510219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:20.510890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:20.523408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:20.573709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 923/1819] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:20.578565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:20.625846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4639-f45a-4b98-9f84-1a5c16bed680 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:20.659372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:20.660444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:20.663663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:20.664095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] PATH_INFO: `/v3/projects/d0c2103ce63f4bdc976b438074a1a0ab/users/326a08ef92664517a11e1df7499deee9/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:20.709083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 897/1820] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:20.730190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c2103ce63f4bdc976b438074a1a0ab, user_id=326a08ef92664517a11e1df7499deee9, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:20.733383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:20.821326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:20.821594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:20.821850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:20.825277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 924/1821] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:20 2022] PUT /identity/v3/projects/d0c2103ce63f4bdc976b438074a1a0ab/users/326a08ef92664517a11e1df7499deee9/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:20.906493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:20.909209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:20.977242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] MFA Rules not processed for user `326a08ef92664517a11e1df7499deee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:20.983882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:21.226174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:21.247122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9beb0-219c-4892-9108-f8ede3e11611 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:21.272374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:21.273222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.273611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.274039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] PATH_INFO: `/v3/users/5f49d3d42cd44838a7fd12d94a4f089a` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.277022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f49d3d42cd44838a7fd12d94a4f089a)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:21.280118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:21.291842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 898/1822] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:21.292830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:21.295727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:21.464548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b6ef2-cad9-4620-89f9-db78875f4b88 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:21.473964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539685dc-4458-48d8-854b-2fdc5f3f2124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:21.474570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539685dc-4458-48d8-854b-2fdc5f3f2124 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.474896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539685dc-4458-48d8-854b-2fdc5f3f2124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.475200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539685dc-4458-48d8-854b-2fdc5f3f2124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.499835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539685dc-4458-48d8-854b-2fdc5f3f2124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:21.501306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539685dc-4458-48d8-854b-2fdc5f3f2124 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:21.518069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f49d3d42cd44838a7fd12d94a4f089a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:21.528585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 899/1823] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:21.530438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90595-2ba8-4663-8134-86242af7657a None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.531116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90595-2ba8-4663-8134-86242af7657a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.531711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90595-2ba8-4663-8134-86242af7657a None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.534272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 900/1824] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:31:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:31:21.535555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd90595-2ba8-4663-8134-86242af7657a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:21.540612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f49d3d42cd44838a7fd12d94a4f089a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:21.541637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f49d3d42cd44838a7fd12d94a4f089a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:21.565003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f49d3d42cd44838a7fd12d94a4f089a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:21.572749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd90595-2ba8-4663-8134-86242af7657a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:21.583325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0be41a-4a56-493e-860c-ab60fc7b743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:21.584363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0be41a-4a56-493e-860c-ab60fc7b743c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.584865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0be41a-4a56-493e-860c-ab60fc7b743c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.585347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0be41a-4a56-493e-860c-ab60fc7b743c admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.586333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0be41a-4a56-493e-860c-ab60fc7b743c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:21.588525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0be41a-4a56-493e-860c-ab60fc7b743c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:21.618913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f49d3d42cd44838a7fd12d94a4f089a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:21.659702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 901/1825] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:21 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:21.659702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf50cf3-d3d6-4fee-a3ea-9c8dae199fdc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.659702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf50cf3-d3d6-4fee-a3ea-9c8dae199fdc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.659702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf50cf3-d3d6-4fee-a3ea-9c8dae199fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.663478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 925/1826] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:20 2022] DELETE /identity/v3/users/5f49d3d42cd44838a7fd12d94a4f089a => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:21.668299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:21.681864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:21.707579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cbf50cf3-d3d6-4fee-a3ea-9c8dae199fdc None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:21.713222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cbf50cf3-d3d6-4fee-a3ea-9c8dae199fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:21.757517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:21.884836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 902/1827] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:21.886262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada6f25-41a7-4ef5-a028-32f9d5ac65b1 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.887938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada6f25-41a7-4ef5-a028-32f9d5ac65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.888346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada6f25-41a7-4ef5-a028-32f9d5ac65b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.890518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee3a3b-dd78-4429-b5c6-5fafc3316381 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:21.903068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:21.903723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.904005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.904302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.904919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:21.907652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:21.933051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8ada6f25-41a7-4ef5-a028-32f9d5ac65b1 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:21.934475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8ada6f25-41a7-4ef5-a028-32f9d5ac65b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:21.954397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 903/1828] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:21.957697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 926/1829] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:21 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:21.958160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9693f8-8e08-488f-af58-18184df31164 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:21.958756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9693f8-8e08-488f-af58-18184df31164 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:21.959482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:21.959960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9693f8-8e08-488f-af58-18184df31164 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:21.964020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 904/1830] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:31:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:31:22.001566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b93ec-1351-4b44-8f4a-8edc22c57435 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:22.024087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:22.024727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.025009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.025289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.026139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:22.028261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:22.044490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.045048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.045448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.063593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 927/1831] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:31:22.071336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.085386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:22.086722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:22.105760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70ea7d-8448-406e-9a60-3eed240b6445 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:22.110068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 905/1832] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:22.118234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.119766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:22.142468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:22.143239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.143547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.143806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.144506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:22.146805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:22.189654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 928/1833] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:22.190621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.229927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10aa141-ebec-478e-a491-f3707a399f02 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:22.248408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb67f6-ac08-4197-b743-3921e1354883 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:22.255245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33f955-eafd-4529-926a-91256023c0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:22.255928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:22.256060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f955-eafd-4529-926a-91256023c0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.256414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f955-eafd-4529-926a-91256023c0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.256767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f955-eafd-4529-926a-91256023c0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.263923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.265084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.265376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] PATH_INFO: `/v3/projects/0bebf177056d4f4a9aa7a0b3d4d685c3/users/0fdf0a2f95714d95b48cb717d8e86080/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.278241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33f955-eafd-4529-926a-91256023c0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:22.279869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33f955-eafd-4529-926a-91256023c0c4 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:22.306387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 906/1834] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:22.308021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bebf177056d4f4a9aa7a0b3d4d685c3, user_id=0fdf0a2f95714d95b48cb717d8e86080, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:22.311586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:22.335846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 929/1835] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:22 2022] PUT /identity/v3/projects/0bebf177056d4f4a9aa7a0b3d4d685c3/users/0fdf0a2f95714d95b48cb717d8e86080/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:22.349002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.349378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.350381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.395621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] MFA Rules not processed for user `0fdf0a2f95714d95b48cb717d8e86080`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:22.398702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:22.512521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.515603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 907/1836] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:22.546339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:22.634056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed7e19-2b47-4de3-858a-f9e3c8c22f31 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:22.640061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:22.640845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.641287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.641668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.642339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:22.644578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:22.695406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.710504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 930/1837] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:22 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:22.714872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c2b76-4896-4909-94d1-2a9cf88998ca None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:22.761050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:22.910439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.912805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:22.919468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:22.920496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:22.921086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:22.921465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:22.925533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:22.928475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:22.987300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 908/1838] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:22 2022] POST /identity/v3/users => generated 299 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:22.988249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:22.991597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.006507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596eb81f-aabd-44a8-8741-d267026a3f62 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.013529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.014624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.015204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.015801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] PATH_INFO: `/v3/projects/7e72cf46cf164c06a541eb9afd0714c9` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.019127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e72cf46cf164c06a541eb9afd0714c9)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.021334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.125960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bae61e-a0bd-4021-8dea-6f7d4c772979 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.132510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e72cf46cf164c06a541eb9afd0714c9', 'request_id': 'req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:23.163024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e72cf46cf164c06a541eb9afd0714c9', 'request_id': 'req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:23.180356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e72cf46cf164c06a541eb9afd0714c9', 'request_id': 'req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:23.200398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] The token cache is being invalidate because project 7e72cf46cf164c06a541eb9afd0714c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:23.201097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:23.203558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:23.206648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 909/1839] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:22 2022] DELETE /identity/v3/projects/7e72cf46cf164c06a541eb9afd0714c9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:23.207892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.211764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.263220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.264385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.264724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.265004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.267479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.284341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.286315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.316604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 931/1840] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:23.317595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.323541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.358137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb84f-a9d9-43ca-827d-fb2dbc215b2b service nova] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.373340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.374235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.374687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.375946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.382921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.391111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1311a3-6496-4cd7-9ff2-e267d5f2b837 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.405756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.425099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.425415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.426440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.427610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.431947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.433935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.434870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2785cc8-8cb7-43a6-8015-506a58dc1e76 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.460987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 932/1841] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:23.469813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245b88-3154-4aa2-94f3-94f0a3670363 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.470604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245b88-3154-4aa2-94f3-94f0a3670363 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.471077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd245b88-3154-4aa2-94f3-94f0a3670363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.488035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 910/1842] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:23.489267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.526784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-dd245b88-3154-4aa2-94f3-94f0a3670363 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:23.528349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dd245b88-3154-4aa2-94f3-94f0a3670363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.535372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8360aa5-3dae-43d3-b43d-e085060ceb93 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.553760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 933/1843] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:23.559386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.559976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.560425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.562163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.565288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.567205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.567584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] PATH_INFO: `/v3/projects/c3e27f6e9d9045fbae8eeb5b8dd85d9c/users/962bf064c08d48a8bdbd3ec2fd38a862/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.610387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:23.612932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:23.617590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e27f6e9d9045fbae8eeb5b8dd85d9c, user_id=962bf064c08d48a8bdbd3ec2fd38a862, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.623448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.634819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 934/1844] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:23.640181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.651771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:23.667790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff5c3e-2154-493a-a1f1-abfd6c0dc5ef None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.670145 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 911/1845] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:23 2022] PUT /identity/v3/projects/c3e27f6e9d9045fbae8eeb5b8dd85d9c/users/962bf064c08d48a8bdbd3ec2fd38a862/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:23.671192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.712951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:23.806685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.807454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.807740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.808070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.809193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:23.809559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.811314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.845262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 935/1846] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:31:23.850418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.856176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bdfa7b-95dd-4eeb-861b-2b2e8e23c09b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.883858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135800-bd21-41fb-b936-ec54dd1dddd6 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.884332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.886610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.887440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.889246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.891799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.895407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.903004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.903834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.904119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.904420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:23.905157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:23.907256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:23.943776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 912/1847] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:23.948068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:23.973599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91870c-cc2e-442d-b39f-c28e4831f34b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:23.993296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec70f03-2429-4890-bbaa-61a0905121ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:23.994174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70f03-2429-4890-bbaa-61a0905121ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:23.994463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70f03-2429-4890-bbaa-61a0905121ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:23.994746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec70f03-2429-4890-bbaa-61a0905121ff admin admin] PATH_INFO: `/v3/projects/c3e27f6e9d9045fbae8eeb5b8dd85d9c/users/962bf064c08d48a8bdbd3ec2fd38a862/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.003949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 936/1848] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:23 2022] POST /identity/v3/projects => generated 359 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:31:24.010841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70f03-2429-4890-bbaa-61a0905121ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e27f6e9d9045fbae8eeb5b8dd85d9c, user_id=962bf064c08d48a8bdbd3ec2fd38a862, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:24.012356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:24.015783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec70f03-2429-4890-bbaa-61a0905121ff admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:24.057681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 913/1849] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:23 2022] PUT /identity/v3/projects/c3e27f6e9d9045fbae8eeb5b8dd85d9c/users/962bf064c08d48a8bdbd3ec2fd38a862/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:24.072289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.072582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.072864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.183398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] MFA Rules not processed for user `962bf064c08d48a8bdbd3ec2fd38a862`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:24.188508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:24.390649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a52646-091c-470a-aaf0-12bf45927d69 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:24.512330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 914/1850] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:24.523643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:24.529198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:24.648478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:24.648478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.648478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.648478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.648902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:24.649331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:24.681917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 937/1851] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:24 2022] POST /identity/v3/users => generated 280 bytes in 672 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:24.687540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:24.713152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741f63-8f74-424c-9ef3-5ef9108e88b2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:24.715951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be05b-fdef-494a-883b-ebe9aa4a5eb1 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:24.716982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:24.717619 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.717882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.718150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.725995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:24.726870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.727478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.727906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.728746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:24.731531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:24.739909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:24.742946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:24.746152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:24.771600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 938/1852] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:24.771882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 915/1853] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:24.777660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:24.792321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d982713-c155-4d5d-9071-aafc9e4da274 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:24.803268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6004b-1198-4ef9-a607-b95bacc2ca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:24.803268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6004b-1198-4ef9-a607-b95bacc2ca85 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.803268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6004b-1198-4ef9-a607-b95bacc2ca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.803268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6004b-1198-4ef9-a607-b95bacc2ca85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.804568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6004b-1198-4ef9-a607-b95bacc2ca85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:24.812061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6004b-1198-4ef9-a607-b95bacc2ca85 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:24.852394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 939/1854] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:24.865874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:24.894318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca1773-8f1c-469a-8ad1-1072e8cde8c2 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:24.904430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:24.907837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.909597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.909597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] PATH_INFO: `/v3/projects/faff139b8f2a4d70a0bc4ae4076b99e7/users/53bfe36a1b0840debf5613adbb33e933/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.948724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=faff139b8f2a4d70a0bc4ae4076b99e7, user_id=53bfe36a1b0840debf5613adbb33e933, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:24.951452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:24.980404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 916/1855] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:24 2022] PUT /identity/v3/projects/faff139b8f2a4d70a0bc4ae4076b99e7/users/53bfe36a1b0840debf5613adbb33e933/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:24.985564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:24.986137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:24.986431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:24.986712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:24.987371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:24.996676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:25.060683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] MFA Rules not processed for user `53bfe36a1b0840debf5613adbb33e933`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:25.062050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:25.119350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f6838-1c29-4957-87a2-cf74b2fdfd6c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:25.127806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:25.128731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:25.129139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:25.129524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] PATH_INFO: `/v3/projects/23c1adbfe1ff40f09ec058bfd706de9a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:25.131625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c1adbfe1ff40f09ec058bfd706de9a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:25.134627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:25.289527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 940/1856] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:25.291172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:25.302380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c1adbfe1ff40f09ec058bfd706de9a', 'request_id': 'req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.330508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c1adbfe1ff40f09ec058bfd706de9a', 'request_id': 'req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.334723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ae59d-0d6d-419d-8bd3-cc7320757cf9 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:25.343834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c1adbfe1ff40f09ec058bfd706de9a', 'request_id': 'req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.390961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] The token cache is being invalidate because project 23c1adbfe1ff40f09ec058bfd706de9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:25.392141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.394037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.414428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 917/1857] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:24 2022] DELETE /identity/v3/projects/23c1adbfe1ff40f09ec058bfd706de9a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:25.415439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:25.420809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:25.462146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:25.463775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:25.466016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:25.466549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:25.467930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:25.475347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:25.559531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 941/1858] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:25 2022] POST /identity/v3/projects => generated 365 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:25.560882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:25.580373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:25.617759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbc8be-de26-4b3d-863a-85e6f05ce4f2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:25.717127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:25.760741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:25.761549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:25.761925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:25.762294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] PATH_INFO: `/v3/users/bccb1a36b282443d8e136d3c79b52948` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:25.763595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccb1a36b282443d8e136d3c79b52948)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:25.765675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:25.771206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b729a0-95bf-42ce-9eeb-8e35a32ffac7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:25.787631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:25.788410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:25.788568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:25.791137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:25.791728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:25.801172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:25.835207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:25.867068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccb1a36b282443d8e136d3c79b52948'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.880467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 942/1859] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:25 2022] POST /identity/v3/users => generated 289 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:25.884875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:25.897115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccb1a36b282443d8e136d3c79b52948'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.919387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f50c-1b94-4723-850f-5f11449b5282 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:25.925967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccb1a36b282443d8e136d3c79b52948'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.927127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccb1a36b282443d8e136d3c79b52948'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.944417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:25.946166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:25.946453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:25.946729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:25.947435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:25.957808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccb1a36b282443d8e136d3c79b52948'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:25.962123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:25.969210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 918/1860] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:25 2022] DELETE /identity/v3/users/bccb1a36b282443d8e136d3c79b52948 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:25.992217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 943/1861] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:25.993171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:25.995975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:26.243574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:26.246414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:26.325522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb147d8-1ced-44b6-9ed4-7380317e439c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:26.340650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:26.341996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:26.343167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:26.344858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] PATH_INFO: `/v3/projects/c07d9bb62098468e9439c1ce8e648232/users/c7c8024c637a4b4bb4a0371175e4847d/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:26.395214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07d9bb62098468e9439c1ce8e648232, user_id=c7c8024c637a4b4bb4a0371175e4847d, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:26.402531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:26.446023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 919/1862] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:25 2022] PUT /identity/v3/projects/c07d9bb62098468e9439c1ce8e648232/users/c7c8024c637a4b4bb4a0371175e4847d/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:26.450061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:26.465545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde2962-add5-4e64-bf57-0b28f8d01d20 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:26.542470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:26.575284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc031e-8f23-4742-889b-2831c113bd91 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:26.583554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:26.585344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:26.585838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:26.586242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:26.587071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:26.589842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:26.593295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:26.593980 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:26.594306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:26.594580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:26.596612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:26.607714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 920/1863] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:26.613331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:26.638955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f277ac-14ae-4f1c-adb4-12e51e4ee69f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:26.656811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:26.657511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:26.657800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:26.660311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] PATH_INFO: `/v3/projects/c07d9bb62098468e9439c1ce8e648232/users/c7c8024c637a4b4bb4a0371175e4847d/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:26.667602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:26.669214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a206183-7a61-4873-a1c9-121856f8bcdc service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:26.669484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07d9bb62098468e9439c1ce8e648232, user_id=c7c8024c637a4b4bb4a0371175e4847d, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:26.672639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:26.684032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 944/1864] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:26 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:26.732402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 921/1865] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:26 2022] PUT /identity/v3/projects/c07d9bb62098468e9439c1ce8e648232/users/c7c8024c637a4b4bb4a0371175e4847d/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:26.745060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:26.745345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:26.745623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:26.792690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] MFA Rules not processed for user `c7c8024c637a4b4bb4a0371175e4847d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:26.797600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:26.841714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:26.843752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:26.964212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:27.038555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 945/1866] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:27.050317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4267d-bf32-4069-975e-f03f69c9aa96 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:27.053840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:27.055607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:27.065545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:27.066966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:27.067457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:27.067910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] PATH_INFO: `/v3/projects/e8058f9450724038b3aa281b8c35a298` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:27.069566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8058f9450724038b3aa281b8c35a298)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:27.072425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:27.211511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d802ae-e714-4426-9987-8a5f98697c65 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:27.236135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8058f9450724038b3aa281b8c35a298', 'request_id': 'req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.256707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8058f9450724038b3aa281b8c35a298', 'request_id': 'req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.272542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8058f9450724038b3aa281b8c35a298', 'request_id': 'req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.300613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] The token cache is being invalidate because project e8058f9450724038b3aa281b8c35a298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:27.302009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.305545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.312922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 922/1867] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:26 2022] DELETE /identity/v3/projects/e8058f9450724038b3aa281b8c35a298 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:27.314304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:27.318053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:27.319168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:27.319801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:27.320081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:27.320353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:27.322102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:27.467190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:27.473069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0d33e-616e-4a53-bbc6-4c1f29a4761d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:27.488773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:27.489809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:27.490204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:27.492456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] PATH_INFO: `/v3/users/3b9108cf7aed4d54819fb77aa17429ec` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:27.495344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b9108cf7aed4d54819fb77aa17429ec)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:27.497620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:27.499082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:27.499749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:27.510821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 946/1868] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:27.614823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b9108cf7aed4d54819fb77aa17429ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.623510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b9108cf7aed4d54819fb77aa17429ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.638972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:27.640814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:27.642234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b9108cf7aed4d54819fb77aa17429ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.643138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b9108cf7aed4d54819fb77aa17429ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.661231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97634e30-2f9f-47a4-bd99-35478958526d service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:27.663953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b9108cf7aed4d54819fb77aa17429ec'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:27.666179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:27.666916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:27.667303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:27.667584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:27.669953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:27.674778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 923/1869] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:27 2022] DELETE /identity/v3/users/3b9108cf7aed4d54819fb77aa17429ec => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:27.750407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:27.770574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:27.772644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba4ece-cd9a-49d5-a819-ae49ed510f17 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:27.783144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 947/1870] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:28.060650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.062266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:28.137111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8218de-6223-411e-8cf9-c19311582949 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.141573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.142220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.142497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.142768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] PATH_INFO: `/v3/projects/6a5df6c9e73d41029590dabdd23d24dc` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.143938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5df6c9e73d41029590dabdd23d24dc)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.145832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.193375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5df6c9e73d41029590dabdd23d24dc', 'request_id': 'req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:28.198974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5df6c9e73d41029590dabdd23d24dc', 'request_id': 'req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:28.205105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5df6c9e73d41029590dabdd23d24dc', 'request_id': 'req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:28.217313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] The token cache is being invalidate because project 6a5df6c9e73d41029590dabdd23d24dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:28.217613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:28.218246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7ea398a3-1a26-4f8c-8dc0-3883f59016d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:28.224157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 924/1871] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:28 2022] DELETE /identity/v3/projects/6a5df6c9e73d41029590dabdd23d24dc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:28.227005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.227655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.228002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.248305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:28.249560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:28.308407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 948/1872] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:28.317949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.318298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.318584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.332866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:28.339211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:28.341107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:28.351484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 925/1873] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:28.355872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.365621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0326e7-6bc2-4e97-8a35-a5632f7cfa7c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.370268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.370865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.371183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.371461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.372106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:28.372411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.374005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.385593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 949/1874] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:28.390259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.399965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a9488-2409-4551-af34-d0dab237d4a6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.404565 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.405163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.405443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.405723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.406385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.408281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.436735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 926/1875] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:28.441245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.451859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47ad91-c06b-45a7-a350-83d10936342f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.515081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.515702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.516100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.516385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.517090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.518999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.550781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 950/1876] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:28.555634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.579994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859018d-b10a-4bda-8bc8-89501fc4d116 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.590724 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.591467 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.591836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.592179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.592905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.594602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.612326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 927/1877] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:28.613473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.625369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e184f-169e-447b-addd-0eba6aec28d2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.632613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d4ea2-9d60-4e42-9589-963832f17cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.633288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d4ea2-9d60-4e42-9589-963832f17cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.633654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d4ea2-9d60-4e42-9589-963832f17cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.634230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d4ea2-9d60-4e42-9589-963832f17cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.635049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d4ea2-9d60-4e42-9589-963832f17cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.636893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d4ea2-9d60-4e42-9589-963832f17cd6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.649693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 951/1878] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:28.653939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.670254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72442d89-3329-4f0d-9256-4a094b964cb4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.670816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.671607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.671890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.672166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] PATH_INFO: `/v3/projects/67a43e8eedbc4b3bb634cfe65195263c/users/f4dfcdcc67994f429d8d2f995eebc550/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.688979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67a43e8eedbc4b3bb634cfe65195263c, user_id=f4dfcdcc67994f429d8d2f995eebc550, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.691931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.715173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 928/1879] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:28 2022] PUT /identity/v3/projects/67a43e8eedbc4b3bb634cfe65195263c/users/f4dfcdcc67994f429d8d2f995eebc550/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:28.723034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.723359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.723636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.755471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] MFA Rules not processed for user `f4dfcdcc67994f429d8d2f995eebc550`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:28.756722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:28.813669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:28.845934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 952/1880] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:28.851272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.923516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b706b3-7b99-4fd1-8e9a-00fef61f78f3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:28.929761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:28.930099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:28.930399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:28.930676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:28.931679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:28.933266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:28.963673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 929/1881] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:28.967563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:28.986794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171b20d-deb8-4797-ad75-8da38994916d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.055558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.056184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.056472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.056744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.057395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.071361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.087194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 953/1882] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:28 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:29.089778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:29.102585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba63b7d-8415-45dd-8b3b-faf242c47221 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.109325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.109951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.110233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.110505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.111196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.112991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.132371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 930/1883] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:29.136235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:29.149291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924071e-6331-45e6-b5cf-3fa47e989edb admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.154514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.155279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.155514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.155789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] PATH_INFO: `/v3/projects/fbc465853f95478d8dcd68e42cf861d4/users/0dd332ae736a476fb7b0a33f4ca46290/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.180180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc465853f95478d8dcd68e42cf861d4, user_id=0dd332ae736a476fb7b0a33f4ca46290, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.180180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.205675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 954/1884] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:29 2022] PUT /identity/v3/projects/fbc465853f95478d8dcd68e42cf861d4/users/0dd332ae736a476fb7b0a33f4ca46290/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:29.210763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:29.233485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c56b00-88bb-466d-b9cf-b37f7908f48a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.357577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e082d3-8741-4762-a8ee-787cc4021279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.358188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e082d3-8741-4762-a8ee-787cc4021279 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.358463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e082d3-8741-4762-a8ee-787cc4021279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.358728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e082d3-8741-4762-a8ee-787cc4021279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.359424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e082d3-8741-4762-a8ee-787cc4021279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.361019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e082d3-8741-4762-a8ee-787cc4021279 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.375616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 931/1885] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:29.376809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:29.387310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5d5c0-58f6-4504-960f-7ffb6687941c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.392648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.393314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.393599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.393862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] PATH_INFO: `/v3/projects/fbc465853f95478d8dcd68e42cf861d4/users/0dd332ae736a476fb7b0a33f4ca46290/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.395886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc465853f95478d8dcd68e42cf861d4, user_id=0dd332ae736a476fb7b0a33f4ca46290, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.399129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.416336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 955/1886] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:29 2022] PUT /identity/v3/projects/fbc465853f95478d8dcd68e42cf861d4/users/0dd332ae736a476fb7b0a33f4ca46290/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:29.428196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.428502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.428773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.479576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:29.485304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] MFA Rules not processed for user `0dd332ae736a476fb7b0a33f4ca46290`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:29.492076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:29.581829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 932/1887] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:29.595216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:29.597478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:29.658923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29211eb7-ae46-4398-8162-98135fd8874c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.662558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.663188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.663468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.663739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.674730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.676181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.687632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 956/1888] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:29.777290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:29.787696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111ae1a-774e-42cf-ad08-4672fb177c5a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:29.791731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad038e2d-7dfe-4a49-b73d-20da4d2e6c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:29.792354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad038e2d-7dfe-4a49-b73d-20da4d2e6c24 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:29.792663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad038e2d-7dfe-4a49-b73d-20da4d2e6c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:29.792942 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad038e2d-7dfe-4a49-b73d-20da4d2e6c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:29.890987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad038e2d-7dfe-4a49-b73d-20da4d2e6c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:29.892393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad038e2d-7dfe-4a49-b73d-20da4d2e6c24 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:29.902088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 933/1889] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:30.229617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:30.270328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:30.314556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e8a19-865d-4c6a-8818-a2f9f6dedbdd service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:30.320299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:30.320911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:30.321190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:30.321458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] PATH_INFO: `/v3/users/f4dfcdcc67994f429d8d2f995eebc550` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:30.322617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4dfcdcc67994f429d8d2f995eebc550)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:30.324550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:30.401900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4dfcdcc67994f429d8d2f995eebc550'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.411434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4dfcdcc67994f429d8d2f995eebc550'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.412686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4dfcdcc67994f429d8d2f995eebc550'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.421413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4dfcdcc67994f429d8d2f995eebc550'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.439823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4dfcdcc67994f429d8d2f995eebc550'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.447488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 957/1890] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:30 2022] DELETE /identity/v3/users/f4dfcdcc67994f429d8d2f995eebc550 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:30.451815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:30.452315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:30.452395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:30.474088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:30.475866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:30.542571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 934/1891] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:30.553657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:30.555118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:30.617995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a024bfbb-701b-4df1-890f-2c140503b1ac admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:30.622199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519bfb48-e9fd-4962-83ff-3ce10df97db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:30.622814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bfb48-e9fd-4962-83ff-3ce10df97db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:30.623115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bfb48-e9fd-4962-83ff-3ce10df97db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:30.623552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519bfb48-e9fd-4962-83ff-3ce10df97db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:30.634126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519bfb48-e9fd-4962-83ff-3ce10df97db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:30.635537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519bfb48-e9fd-4962-83ff-3ce10df97db3 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:30.647268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 958/1892] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:30.736366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:30.739119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:30.753851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10f0af-0898-47f9-8f52-260d209183b9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:30.759786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:30.760479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:30.761079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:30.761368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] PATH_INFO: `/v3/users/53bfe36a1b0840debf5613adbb33e933` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:30.763759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bfe36a1b0840debf5613adbb33e933)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:30.765765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:30.815043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:30.831793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bfe36a1b0840debf5613adbb33e933'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.840061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bfe36a1b0840debf5613adbb33e933'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.870556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bfe36a1b0840debf5613adbb33e933'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.871496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bfe36a1b0840debf5613adbb33e933'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.892524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bfe36a1b0840debf5613adbb33e933'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:30.901847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 935/1893] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:30 2022] DELETE /identity/v3/users/53bfe36a1b0840debf5613adbb33e933 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:30.906682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:30.907245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:30.907738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:30.932171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:30.933389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:31.001334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:31.020454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 959/1894] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:31.051640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:31.053712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:31.087136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:31.088465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:31.141200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a100-42aa-4cdd-9c8e-90cb03474fce None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:31.169420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:31.172035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:31.175548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:31.176100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] PATH_INFO: `/v3/projects/67a43e8eedbc4b3bb634cfe65195263c` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:31.178535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67a43e8eedbc4b3bb634cfe65195263c)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:31.182358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:31.233065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b557adb-a3a4-4270-80be-624ee2da6d9b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:31.237404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:31.238020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:31.238438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:31.238770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:31.249617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:31.251492 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:31.269239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 936/1895] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:31.281536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67a43e8eedbc4b3bb634cfe65195263c', 'request_id': 'req-547a2e0c-e2f7-42ce-b9bd-4161949864d5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.288869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67a43e8eedbc4b3bb634cfe65195263c', 'request_id': 'req-547a2e0c-e2f7-42ce-b9bd-4161949864d5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.304705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67a43e8eedbc4b3bb634cfe65195263c', 'request_id': 'req-547a2e0c-e2f7-42ce-b9bd-4161949864d5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.325916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] The token cache is being invalidate because project 67a43e8eedbc4b3bb634cfe65195263c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:31.326351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.327127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.333079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:31.335298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 960/1896] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:31 2022] DELETE /identity/v3/projects/67a43e8eedbc4b3bb634cfe65195263c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:31.345200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c2924-b231-4e24-ab41-e78cfb25930b service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:31.371609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:31.373204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:31.431460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:31.432672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:31.433176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:31.433836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] PATH_INFO: `/v3/users/0dd332ae736a476fb7b0a33f4ca46290` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:31.435337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd332ae736a476fb7b0a33f4ca46290)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:31.437795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:31.459999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a2e0c-e2f7-42ce-b9bd-4161949864d5 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:31.464582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:31.465200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:31.465477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:31.465753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:31.468261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:31.506453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd332ae736a476fb7b0a33f4ca46290'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.514271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:31.516466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd332ae736a476fb7b0a33f4ca46290'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.527504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd332ae736a476fb7b0a33f4ca46290'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.529875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd332ae736a476fb7b0a33f4ca46290'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.547081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd332ae736a476fb7b0a33f4ca46290'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.549398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:31.551311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:31.554409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 937/1897] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:31 2022] DELETE /identity/v3/users/0dd332ae736a476fb7b0a33f4ca46290 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:31.563331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 961/1898] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:31.703636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:31.705075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:31.851860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9a9f5-a767-4c88-8667-61f3467cae04 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:31.858534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:31.859272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:31.860516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:31.860991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] PATH_INFO: `/v3/projects/faff139b8f2a4d70a0bc4ae4076b99e7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:31.862511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=faff139b8f2a4d70a0bc4ae4076b99e7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:31.865925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:31.952313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faff139b8f2a4d70a0bc4ae4076b99e7', 'request_id': 'req-1c7fc8c5-7459-4733-a2fb-42313f6de58e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.961266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faff139b8f2a4d70a0bc4ae4076b99e7', 'request_id': 'req-1c7fc8c5-7459-4733-a2fb-42313f6de58e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:31.972138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faff139b8f2a4d70a0bc4ae4076b99e7', 'request_id': 'req-1c7fc8c5-7459-4733-a2fb-42313f6de58e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.002332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] The token cache is being invalidate because project faff139b8f2a4d70a0bc4ae4076b99e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:32.002779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.003608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.004844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.009344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 938/1899] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:31 2022] DELETE /identity/v3/projects/faff139b8f2a4d70a0bc4ae4076b99e7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:32.013260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.019411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.026130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.040913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd78a2-fc8b-4539-8676-5ec469fe97a4 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:32.103945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:32.113329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fc8c5-7459-4733-a2fb-42313f6de58e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:32.122163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:32.122976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.123356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.123701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] PATH_INFO: `/v3/users/c7c8024c637a4b4bb4a0371175e4847d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.125438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c8024c637a4b4bb4a0371175e4847d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:32.128354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:32.131910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:32.165223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:32.166474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.166474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.166671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] PATH_INFO: `/v3/projects/fbc465853f95478d8dcd68e42cf861d4` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.168561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc465853f95478d8dcd68e42cf861d4)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:32.171451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:32.217974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c8024c637a4b4bb4a0371175e4847d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.235174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c8024c637a4b4bb4a0371175e4847d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.246633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c8024c637a4b4bb4a0371175e4847d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.247363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c8024c637a4b4bb4a0371175e4847d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.263671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e906b83f-b0c7-4b5a-8e17-0126511b79ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c8024c637a4b4bb4a0371175e4847d'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.268810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc465853f95478d8dcd68e42cf861d4', 'request_id': 'req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.276072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 939/1900] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:32 2022] DELETE /identity/v3/users/c7c8024c637a4b4bb4a0371175e4847d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:32.282565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc465853f95478d8dcd68e42cf861d4', 'request_id': 'req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.291439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc465853f95478d8dcd68e42cf861d4', 'request_id': 'req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.310417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] The token cache is being invalidate because project fbc465853f95478d8dcd68e42cf861d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:32.310860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.311690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.315618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 962/1901] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:31 2022] DELETE /identity/v3/projects/fbc465853f95478d8dcd68e42cf861d4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:32.320569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e10277-2add-430b-b278-72afd8c04b0c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.320860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e10277-2add-430b-b278-72afd8c04b0c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.321140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e10277-2add-430b-b278-72afd8c04b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.328634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.331118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.359633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f5e10277-2add-430b-b278-72afd8c04b0c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:32.360904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f5e10277-2add-430b-b278-72afd8c04b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.423816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8d6cf-76ef-406d-afd1-a247d93a45e1 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:32.462647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 940/1902] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:32.471211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.471810 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.472207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.510501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:32.511506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.512287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.512713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] PATH_INFO: `/v3/users/00b5b80b8d0140bb86ac003362f7e035` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.514239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b5b80b8d0140bb86ac003362f7e035)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:32.516454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:32.519608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:32.525849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.543348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:32.545117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 941/1903] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:32.549985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.569399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa07e6-8559-4b67-af44-6e5b1e52eb71 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:32.579537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:32.580478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.580478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.580832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.582738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:32.582871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:32.586702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:32.618305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 942/1904] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:31:32.619670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b5b80b8d0140bb86ac003362f7e035'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.621946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.638233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea24cb5-bae3-40e1-a2b0-5f61b2fdcd81 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:32.643914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b5b80b8d0140bb86ac003362f7e035'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.644469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b5b80b8d0140bb86ac003362f7e035'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.645441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:32.648194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.648504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.648837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.649609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:32.652043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:32.659970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b5b80b8d0140bb86ac003362f7e035'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.690267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e054ad4d-9596-4a0c-9ffa-49b345b52540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b5b80b8d0140bb86ac003362f7e035'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:32.704630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 963/1905] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:32 2022] DELETE /identity/v3/users/00b5b80b8d0140bb86ac003362f7e035 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:32.709203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.709610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.709984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.711583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 943/1906] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:32 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:32.717376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.721048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.749365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:32.765887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.826574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:32.890621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8165d28-f1c6-4b4f-8c14-54a46a67e652 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:32.899326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:32.900033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:32.900469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:32.900754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:32.901630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:32.903773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:32.954434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 944/1907] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:32 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:31:32.958445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.966924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 964/1908] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:32.978456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:32.981721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:32.989513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3947e6-81cc-4386-85d6-34b530b5c053 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.002987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.004301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.005663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.006367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.007557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.010051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.026445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 945/1909] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:33.030495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.047790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644f0e0-619c-4bf6-9f43-245ebafdd912 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.057606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.060001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.060793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.061136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.062373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.064792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.082016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 946/1910] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:33.087233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.097773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0c646-132b-449b-8cb7-fbb0b7786fcd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.102582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83305cfb-fd4e-49e3-8749-40d16d5ac03a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.103276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83305cfb-fd4e-49e3-8749-40d16d5ac03a service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.103560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83305cfb-fd4e-49e3-8749-40d16d5ac03a service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.103833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83305cfb-fd4e-49e3-8749-40d16d5ac03a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.104264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cd5fc-3693-4c31-83ee-7daf0529d9f0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.112196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.113156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.113268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.113670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] PATH_INFO: `/v3/projects/b4bdc8c8bf9446259b3ace5df1bc1427/users/180412e877a041c48b56346456a7c135/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.119958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83305cfb-fd4e-49e3-8749-40d16d5ac03a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.122326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83305cfb-fd4e-49e3-8749-40d16d5ac03a service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.135023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bdc8c8bf9446259b3ace5df1bc1427, user_id=180412e877a041c48b56346456a7c135, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.137336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 965/1911] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:33.138091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.160436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 947/1912] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:33 2022] PUT /identity/v3/projects/b4bdc8c8bf9446259b3ace5df1bc1427/users/180412e877a041c48b56346456a7c135/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:33.173323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.174306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.175587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.240958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.243002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.248087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] MFA Rules not processed for user `180412e877a041c48b56346456a7c135`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:33.249330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.287553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:33.383771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 966/1913] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:33.397310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.400154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.432517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450e17b-1312-4640-bc7c-28c56d20233f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.444098 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.445692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.446101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.449489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] PATH_INFO: `/v3/projects/c07d9bb62098468e9439c1ce8e648232` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.450940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07d9bb62098468e9439c1ce8e648232)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.454385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.483190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73eac76-5124-4e9d-815a-c49cde27c467 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.487548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.488165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.488443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.488714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.499898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.502319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.513951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 967/1914] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:33.536421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07d9bb62098468e9439c1ce8e648232', 'request_id': 'req-67ddb4c5-4dde-479a-9a01-b32c9030e441'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.546549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07d9bb62098468e9439c1ce8e648232', 'request_id': 'req-67ddb4c5-4dde-479a-9a01-b32c9030e441'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.552238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.556926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07d9bb62098468e9439c1ce8e648232', 'request_id': 'req-67ddb4c5-4dde-479a-9a01-b32c9030e441'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.567721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc939b5-8c11-413d-896a-21560aea2aca service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.575267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] The token cache is being invalidate because project c07d9bb62098468e9439c1ce8e648232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:33.575561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.576213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67ddb4c5-4dde-479a-9a01-b32c9030e441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.581424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 948/1915] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:33 2022] DELETE /identity/v3/projects/c07d9bb62098468e9439c1ce8e648232 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:33.582719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.583008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.583313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.619731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:33.621071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.630923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:33.638949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.639746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.640052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.640345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] PATH_INFO: `/v3/users/180412e877a041c48b56346456a7c135` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.641651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=180412e877a041c48b56346456a7c135)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.643813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.695951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 949/1916] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:33.696924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.698401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.700827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '180412e877a041c48b56346456a7c135'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.709787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '180412e877a041c48b56346456a7c135'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.710422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '180412e877a041c48b56346456a7c135'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.711070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a55de-a639-4f81-9c77-8c9f5eef07a5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.715991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.716630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.716914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.717189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] PATH_INFO: `/v3/projects/822de5994c174b92bb31327f1ed2dd00` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.717917 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '180412e877a041c48b56346456a7c135'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.718407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=822de5994c174b92bb31327f1ed2dd00)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.720404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.722628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:33.733080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b3b1eb7b-3efe-4165-a301-eae88dc9b1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '180412e877a041c48b56346456a7c135'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.739725 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 968/1917] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:33 2022] DELETE /identity/v3/users/180412e877a041c48b56346456a7c135 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:33.743661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.744362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.744362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.773282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:33.774512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.782261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822de5994c174b92bb31327f1ed2dd00', 'request_id': 'req-48bc7721-79b8-4424-9e3b-20808cfc1349'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.791871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822de5994c174b92bb31327f1ed2dd00', 'request_id': 'req-48bc7721-79b8-4424-9e3b-20808cfc1349'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.801100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822de5994c174b92bb31327f1ed2dd00', 'request_id': 'req-48bc7721-79b8-4424-9e3b-20808cfc1349'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.822614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] The token cache is being invalidate because project 822de5994c174b92bb31327f1ed2dd00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:33.822908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.823987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-48bc7721-79b8-4424-9e3b-20808cfc1349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.828818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 950/1918] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:33 2022] DELETE /identity/v3/projects/822de5994c174b92bb31327f1ed2dd00 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:33.828818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.828818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.828818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.847037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 969/1919] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:33.851116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.851116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.853285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:33.854527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.866588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7581b-47f2-4ed1-bee6-6719e9f68ba4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.868315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 951/1920] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:33.869263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.880260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d9483-faa4-428a-adc1-7373384ac766 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.880598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.881481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.881726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.882061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] PATH_INFO: `/v3/users/2a8365b8a71f456a8167712dc565fe07` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.883300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8365b8a71f456a8167712dc565fe07)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.885595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.891763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.892518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.892868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.893360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.894088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:33.894502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:33.896815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:33.913100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 952/1921] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:33.913100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.913100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.942896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8365b8a71f456a8167712dc565fe07'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.954500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8365b8a71f456a8167712dc565fe07'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.955368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8365b8a71f456a8167712dc565fe07'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.964706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8365b8a71f456a8167712dc565fe07'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.977692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8365b8a71f456a8167712dc565fe07'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:33.980065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765f01b-a5c9-474b-8726-01d6322c2c12 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:33.983796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:33.984420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:33.984717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:33.984984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:33.985945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 970/1922] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:33 2022] DELETE /identity/v3/users/2a8365b8a71f456a8167712dc565fe07 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:33.987113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:33.989520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:33.991154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:34.000205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.003168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.018468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 953/1923] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:34.133448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:34.138061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36735f9-3f98-460d-8151-0d96df009ee3 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:34.160146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:34.160885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.161242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.161588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.162393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.164449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.228906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 971/1924] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:33 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:34.233231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.247184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97641053-8aba-476b-95fe-74570a4b6a2a service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:34.285613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:34.388832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:34.389497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.389775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.390064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.390795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.392864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.437648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 954/1925] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:34 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:31:34.439285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.454119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.455900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:34.456689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bec174-d87c-4408-b033-09d923241884 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:34.468243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:34.469132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.469481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.469823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.473405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792dc2a2-d4f9-4542-b34c-726f9a91a81c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:34.474161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.476620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.486771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:34.488095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.488095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.488869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] PATH_INFO: `/v3/projects/b4bdc8c8bf9446259b3ace5df1bc1427` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.490554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bdc8c8bf9446259b3ace5df1bc1427)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.493287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.503082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 972/1926] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:34.504322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.530888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141eef34-c892-4448-9373-a9c7037df832 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:34.549450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:34.550184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.550540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.551128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.552131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.553871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.569662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bdc8c8bf9446259b3ace5df1bc1427', 'request_id': 'req-bc926114-bd32-47ba-a30f-1d8d1c59373c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:34.576380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 973/1927] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:34.576567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bdc8c8bf9446259b3ace5df1bc1427', 'request_id': 'req-bc926114-bd32-47ba-a30f-1d8d1c59373c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:34.576865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.585116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bdc8c8bf9446259b3ace5df1bc1427', 'request_id': 'req-bc926114-bd32-47ba-a30f-1d8d1c59373c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:34.601446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6d922-c123-4340-9319-6f81dea27329 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:34.603798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] The token cache is being invalidate because project b4bdc8c8bf9446259b3ace5df1bc1427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:34.604187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:34.605387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-bc926114-bd32-47ba-a30f-1d8d1c59373c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:34.607966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 955/1928] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:34 2022] DELETE /identity/v3/projects/b4bdc8c8bf9446259b3ace5df1bc1427 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:34.614240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d043-9163-467f-ac3d-443d33e29c8c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.614741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d043-9163-467f-ac3d-443d33e29c8c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.617059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497d043-9163-467f-ac3d-443d33e29c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.696147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4497d043-9163-467f-ac3d-443d33e29c8c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:34.697508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4497d043-9163-467f-ac3d-443d33e29c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:34.739763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162091a-9797-43d8-8ac1-d826ba3b8fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:34.741386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162091a-9797-43d8-8ac1-d826ba3b8fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.741730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162091a-9797-43d8-8ac1-d826ba3b8fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.742656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162091a-9797-43d8-8ac1-d826ba3b8fa8 admin admin] PATH_INFO: `/v3/projects/ffb779bfa1034442b023362d56bab4ff/users/6dd452efd6a8471fb6e58171882f3067/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.763474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162091a-9797-43d8-8ac1-d826ba3b8fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb779bfa1034442b023362d56bab4ff, user_id=6dd452efd6a8471fb6e58171882f3067, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:34.768084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162091a-9797-43d8-8ac1-d826ba3b8fa8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:34.786520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 974/1929] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:34 2022] PUT /identity/v3/projects/ffb779bfa1034442b023362d56bab4ff/users/6dd452efd6a8471fb6e58171882f3067/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:34.798122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.798406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.798684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.833992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 956/1930] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:34.842663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:34.843043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:34.843414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:34.852492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] MFA Rules not processed for user `6dd452efd6a8471fb6e58171882f3067`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:34.853733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:34.890161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:34.891459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:34.934625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:34.977088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 975/1931] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:34.978088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.982089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:34.994715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 957/1932] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:34.994715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:34.994715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.001242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cd2c1-ee8e-46d0-af39-11624f84dcd6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.008320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.009159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.009330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82031a3-26cb-4b1c-b76c-9e12a7770aaf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.009674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.010036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] PATH_INFO: `/v3/users/326a08ef92664517a11e1df7499deee9` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.011313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=326a08ef92664517a11e1df7499deee9)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.013682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.014076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.014696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.015034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.015371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.016008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.017931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.050185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 958/1933] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:34 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:35.052141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.055224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.075934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:35.118722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326a08ef92664517a11e1df7499deee9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.131605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326a08ef92664517a11e1df7499deee9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.133160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326a08ef92664517a11e1df7499deee9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.142704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326a08ef92664517a11e1df7499deee9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.156013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f896d50-44ef-435f-8c6c-2f027be3c1b5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.159114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326a08ef92664517a11e1df7499deee9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.161000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.161761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.161995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.162348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] PATH_INFO: `/v3/projects/7627faaaad42449daf5f45ab212ed7d1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.163480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7627faaaad42449daf5f45ab212ed7d1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.165415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.166661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 976/1934] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:34 2022] DELETE /identity/v3/users/326a08ef92664517a11e1df7499deee9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:35.170496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.171958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.218042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7627faaaad42449daf5f45ab212ed7d1', 'request_id': 'req-4a7baabc-d67a-47f2-93bd-0136ec1b602e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.223659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7627faaaad42449daf5f45ab212ed7d1', 'request_id': 'req-4a7baabc-d67a-47f2-93bd-0136ec1b602e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.229984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7627faaaad42449daf5f45ab212ed7d1', 'request_id': 'req-4a7baabc-d67a-47f2-93bd-0136ec1b602e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.244628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] The token cache is being invalidate because project 7627faaaad42449daf5f45ab212ed7d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:35.245014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.245251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ba9bf-2300-42a1-b94c-4f322ceddc83 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.245675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:35.247318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 959/1935] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:35 2022] DELETE /identity/v3/projects/7627faaaad42449daf5f45ab212ed7d1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:35.248068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.249389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.296275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:35.306241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.306868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.307199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.307460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.308134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:35.308429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.310086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7644403-345b-4653-9b1c-50b203e6a411 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.317721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7baabc-d67a-47f2-93bd-0136ec1b602e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.326007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 977/1936] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:35.326184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e259614-e0fb-4c0b-a14e-4fdeafdf161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.326184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e259614-e0fb-4c0b-a14e-4fdeafdf161d admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.326184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e259614-e0fb-4c0b-a14e-4fdeafdf161d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.326184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e259614-e0fb-4c0b-a14e-4fdeafdf161d admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.326485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.326485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.326485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.326683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e259614-e0fb-4c0b-a14e-4fdeafdf161d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.326752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e259614-e0fb-4c0b-a14e-4fdeafdf161d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.345902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:35.347145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.349249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 960/1937] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:35.354015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.354307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.354591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.358809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 978/1938] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:35.358809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.358809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.369518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8baeb5e-9c09-473a-9e29-b69800b0692e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.375354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:35.376658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.377876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.378496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.378770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.379118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.379762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.381716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.385298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 961/1939] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:35.385894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.395743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d024f-3fe4-4a59-9e25-77f988844c4d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.401199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917012-8526-4142-a829-d74d3311526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.402026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917012-8526-4142-a829-d74d3311526e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.402310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917012-8526-4142-a829-d74d3311526e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.402582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917012-8526-4142-a829-d74d3311526e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.403293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917012-8526-4142-a829-d74d3311526e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.404974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917012-8526-4142-a829-d74d3311526e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.410055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 979/1940] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:35.410937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.413118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.423236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 962/1941] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:35.423236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.423236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.423236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.434273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:35.441912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:35.443798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:35.475891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f9706-e4f8-4b2f-9a6c-c93f0a83f10d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.479551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.480149 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.480430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.480702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.503023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 963/1942] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:35.503879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.514485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f1289-59ff-4d4f-ad7c-a6722f0bac8a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.519226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.519833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.520119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.520391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.521028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.523553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.545941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 964/1943] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/users => generated 299 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:35.547355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.551728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.553118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.558286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0efa1a-47fc-4a2b-b38f-0a724d948929 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.562789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 980/1944] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:35.563247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.563928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.564255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.564539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] PATH_INFO: `/v3/projects/3ea5d45056244861b65a2dc1e8972858/users/7f7dbf195f7e428192ae7d4c5c03287f/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.569352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.580460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d37f5-2eb6-440a-b09e-675a4cb1e8c3 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.580832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea5d45056244861b65a2dc1e8972858, user_id=7f7dbf195f7e428192ae7d4c5c03287f, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.584260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.585792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.586464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.586744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.587018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.587712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:35.588031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.590076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.609090 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 965/1945] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:35 2022] PUT /identity/v3/projects/3ea5d45056244861b65a2dc1e8972858/users/7f7dbf195f7e428192ae7d4c5c03287f/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:35.610343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.612239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 981/1946] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:35.613753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.630041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1b65-bfa0-414e-a333-5acb97d6f8d1 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.717114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed00b03-5331-4253-a239-eaec68a10fe6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.733277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.733920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.734203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.738297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.738528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.738961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.739301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.739431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.739585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.740283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.740993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.742048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.756200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 982/1947] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:35.758654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.775335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 966/1948] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:35.775564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e6aec-5d1b-4eb7-8be0-9556232989df admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.779214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.783296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.783930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.784213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.784558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.785393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.787766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.803938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584606e-4ddf-4eae-9d91-bdd2a42a413d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.810099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.811025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.811554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.811801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] PATH_INFO: `/v3/projects/3ea5d45056244861b65a2dc1e8972858/users/7f7dbf195f7e428192ae7d4c5c03287f/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.813968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ea5d45056244861b65a2dc1e8972858, user_id=7f7dbf195f7e428192ae7d4c5c03287f, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.817017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.824246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:35.838649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 967/1949] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:35 2022] PUT /identity/v3/projects/3ea5d45056244861b65a2dc1e8972858/users/7f7dbf195f7e428192ae7d4c5c03287f/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:35.839898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.842011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 983/1950] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:35.849834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:35.898498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:35.933972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1509b-5734-42b0-bad3-42dc3e5d5e20 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.939202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647e7a-d2d9-40e5-a321-1cbd31e7b43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.939969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647e7a-d2d9-40e5-a321-1cbd31e7b43b admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.940472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647e7a-d2d9-40e5-a321-1cbd31e7b43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.940981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647e7a-d2d9-40e5-a321-1cbd31e7b43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.942680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647e7a-d2d9-40e5-a321-1cbd31e7b43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.944141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647e7a-d2d9-40e5-a321-1cbd31e7b43b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:35.953891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc32c77-f496-441d-aeef-6c30d422afbc admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:35.962842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 968/1951] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:35.964595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.964945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.965278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.968950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:35.969582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:35.969965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:35.970115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:35.970745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:35.972720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.000739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] MFA Rules not processed for user `7f7dbf195f7e428192ae7d4c5c03287f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:36.002374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.026097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 984/1952] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:36.027044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.047950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50be81-fc98-4227-a042-7cac104fe803 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.054222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.054896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.055204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.055481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] PATH_INFO: `/v3/projects/caca9db1e0174372966bd1b6b8548619/users/1bd41e80e37a40b98f6a01b947692181/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.072250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caca9db1e0174372966bd1b6b8548619, user_id=1bd41e80e37a40b98f6a01b947692181, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.074967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.102575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 985/1953] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:36 2022] PUT /identity/v3/projects/caca9db1e0174372966bd1b6b8548619/users/1bd41e80e37a40b98f6a01b947692181/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:36.102575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.102864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 969/1954] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:36.102864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.106171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.117054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c6cf6-2a9d-41ed-8574-ea3bf4159b8d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.188472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbf371-0c60-4090-8573-fab8b84657e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.189195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbf371-0c60-4090-8573-fab8b84657e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.189551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbf371-0c60-4090-8573-fab8b84657e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.190115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbf371-0c60-4090-8573-fab8b84657e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.190605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbf371-0c60-4090-8573-fab8b84657e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.192784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbf371-0c60-4090-8573-fab8b84657e7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.196989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5cba09-ca4b-42c8-a385-a33649b3b871 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.202028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.202860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.203404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.203845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] PATH_INFO: `/v3/projects/d0c2103ce63f4bdc976b438074a1a0ab` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.205329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 970/1955] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:36.206637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.206984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.207315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.209215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c2103ce63f4bdc976b438074a1a0ab)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.211956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.233783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:36.256239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] MFA Rules not processed for user `1bd41e80e37a40b98f6a01b947692181`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:36.257376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.301508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c2103ce63f4bdc976b438074a1a0ab', 'request_id': 'req-e427d438-bc9e-41c1-8f05-6bf285425d6a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.311629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c2103ce63f4bdc976b438074a1a0ab', 'request_id': 'req-e427d438-bc9e-41c1-8f05-6bf285425d6a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.324711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c2103ce63f4bdc976b438074a1a0ab', 'request_id': 'req-e427d438-bc9e-41c1-8f05-6bf285425d6a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.352155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] The token cache is being invalidate because project d0c2103ce63f4bdc976b438074a1a0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:36.352744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.354426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.359940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 986/1956] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:36 2022] DELETE /identity/v3/projects/d0c2103ce63f4bdc976b438074a1a0ab => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:36.361748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.363761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.366868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 971/1957] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:36.368555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.385299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f2ee4-3d94-4057-b5a8-e804e7532099 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.440527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:36.461655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.462493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.462884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.463343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.464234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.466127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.478258 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 972/1958] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:36.479266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.485000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.485466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427d438-bc9e-41c1-8f05-6bf285425d6a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.489268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.490504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.490790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.491063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.493006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.499959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c390ccf-eaf5-4ff3-b51d-a401f295f33c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.506209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.506846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.507178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.507460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] PATH_INFO: `/v3/users/0fdf0a2f95714d95b48cb717d8e86080` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.508622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdf0a2f95714d95b48cb717d8e86080)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.510583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.566766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.568189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.570897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdf0a2f95714d95b48cb717d8e86080'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.577994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 987/1959] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:36.578882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.580283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.580661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdf0a2f95714d95b48cb717d8e86080'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.592796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdf0a2f95714d95b48cb717d8e86080'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.594348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdf0a2f95714d95b48cb717d8e86080'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.595070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df0715-7afe-4e65-8516-fc54b748e19f service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.600978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.601758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.602386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.602386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.605439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.615860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdf0a2f95714d95b48cb717d8e86080'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:36.625303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 973/1960] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:36 2022] DELETE /identity/v3/users/0fdf0a2f95714d95b48cb717d8e86080 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:36.625940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:36.627384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.667204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:36.725680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.729602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08f97f-5030-45ce-845b-3cf64a05ec03 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.752048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 988/1961] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:36 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:36.756089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f79b-7c1d-4190-b8b9-ae1839e3a847 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.756456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f79b-7c1d-4190-b8b9-ae1839e3a847 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.757129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961f79b-7c1d-4190-b8b9-ae1839e3a847 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.760411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 989/1962] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:31:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:31:36.773631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eeb7c-800d-4f3e-af64-ae7cd542625f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:36.781969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.782257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.782535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.797100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50261028-7bc9-4977-a5dc-93002060b727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:36.797794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261028-7bc9-4977-a5dc-93002060b727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.798073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261028-7bc9-4977-a5dc-93002060b727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.798348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50261028-7bc9-4977-a5dc-93002060b727 admin admin] PATH_INFO: `/v3/projects/2d566366c64641a69e791f7208532828/users/dfff7de270c140b2800a8b53c1157fc3/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.824139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:36.829858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:36.837762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50261028-7bc9-4977-a5dc-93002060b727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d566366c64641a69e791f7208532828, user_id=dfff7de270c140b2800a8b53c1157fc3, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:36.843911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50261028-7bc9-4977-a5dc-93002060b727 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:36.875759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 974/1963] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:36 2022] PUT /identity/v3/projects/2d566366c64641a69e791f7208532828/users/dfff7de270c140b2800a8b53c1157fc3/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:36.886591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:36.887719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:36.888209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:36.969636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] MFA Rules not processed for user `dfff7de270c140b2800a8b53c1157fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:36.974438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:37.026608 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 990/1964] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:31:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:37.053880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.055810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:37.079920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139909b2-ce5f-4de2-aa26-a66986a04c95 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.101594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:37.201308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 975/1965] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:37.202353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.203797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:37.241531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.242371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.242652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.243260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.253737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.256317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.276638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 991/1966] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:31:37.278231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.388600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3f767-52ad-41d0-9de3-9743faeb4e92 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.400252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390bafb-bc40-41a6-92aa-120462df8aae service nova] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.417318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.418369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.418816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.419396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] PATH_INFO: `/v3/projects/0bebf177056d4f4a9aa7a0b3d4d685c3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.421102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bebf177056d4f4a9aa7a0b3d4d685c3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.425707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.428044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.431373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.431787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.432692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.432908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.434874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.498088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 992/1967] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:37 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:37.502878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.541470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:37.582876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bebf177056d4f4a9aa7a0b3d4d685c3', 'request_id': 'req-95d9e738-6d45-4077-ace4-4ae55e700dd8'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.594596 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bebf177056d4f4a9aa7a0b3d4d685c3', 'request_id': 'req-95d9e738-6d45-4077-ace4-4ae55e700dd8'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.605111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bebf177056d4f4a9aa7a0b3d4d685c3', 'request_id': 'req-95d9e738-6d45-4077-ace4-4ae55e700dd8'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.628997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] The token cache is being invalidate because project 0bebf177056d4f4a9aa7a0b3d4d685c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:37.629300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.631840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.637036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 976/1968] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:37 2022] DELETE /identity/v3/projects/0bebf177056d4f4a9aa7a0b3d4d685c3 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:37.641992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.643551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:37.651708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e4be-8397-4b01-a1c0-eca970583411 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.699741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:37.749988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.750785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.752394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.752678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.753330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.756612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9e738-6d45-4077-ace4-4ae55e700dd8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.757018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.767593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.768485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.768900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.769526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] PATH_INFO: `/v3/users/962bf064c08d48a8bdbd3ec2fd38a862` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.772772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=962bf064c08d48a8bdbd3ec2fd38a862)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.774986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.784703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 993/1969] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:37 2022] POST /identity/v3/users => generated 299 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:37.789655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.793761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:37.812969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86403c8c-68d8-4249-a57e-9a58c509a82a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.818052 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.820429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.820952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.821360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.822794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.824592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.840560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 994/1970] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:37.848976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.872562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfef010-9a6a-43dc-9070-9ff5331e642f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:37.884503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '962bf064c08d48a8bdbd3ec2fd38a862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.889871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:37.891942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:37.892312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:37.892749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] PATH_INFO: `/v3/projects/dbaabd61abfb42289e1ac64a793298d7/users/8faac3c22a3a4493bae765fc0feee140/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:37.901072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '962bf064c08d48a8bdbd3ec2fd38a862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.915037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbaabd61abfb42289e1ac64a793298d7, user_id=8faac3c22a3a4493bae765fc0feee140, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:37.917402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '962bf064c08d48a8bdbd3ec2fd38a862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.919037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:37.920254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '962bf064c08d48a8bdbd3ec2fd38a862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.942736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '962bf064c08d48a8bdbd3ec2fd38a862'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:37.953624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 995/1971] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:37 2022] PUT /identity/v3/projects/dbaabd61abfb42289e1ac64a793298d7/users/8faac3c22a3a4493bae765fc0feee140/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:37.956126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 977/1972] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:37 2022] DELETE /identity/v3/users/962bf064c08d48a8bdbd3ec2fd38a862 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:37.957638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:37.960185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.052733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4ac7d-5f15-48ed-926b-44d43d712855 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:38.146438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97746761-480a-4ef2-b1e3-7bfb8c15371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:38.147067 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97746761-480a-4ef2-b1e3-7bfb8c15371d admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.147380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97746761-480a-4ef2-b1e3-7bfb8c15371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.147797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97746761-480a-4ef2-b1e3-7bfb8c15371d admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.148501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97746761-480a-4ef2-b1e3-7bfb8c15371d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:38.150185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97746761-480a-4ef2-b1e3-7bfb8c15371d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:38.164112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 996/1973] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:38.168269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:38.180410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c44ce-f7a0-4090-b393-3ca87ba68436 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:38.190471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:38.190471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.190471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.190471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] PATH_INFO: `/v3/projects/dbaabd61abfb42289e1ac64a793298d7/users/8faac3c22a3a4493bae765fc0feee140/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.191244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbaabd61abfb42289e1ac64a793298d7, user_id=8faac3c22a3a4493bae765fc0feee140, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:38.192400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:38.215449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 978/1974] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:38 2022] PUT /identity/v3/projects/dbaabd61abfb42289e1ac64a793298d7/users/8faac3c22a3a4493bae765fc0feee140/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:38.229494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.230092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.230586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.266270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] MFA Rules not processed for user `8faac3c22a3a4493bae765fc0feee140`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:38.267925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.281624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:38.352265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:38.353998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.366786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 997/1975] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:38.379569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:38.381963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.401494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:38.460838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276b0b9-2a45-4ca7-851a-f4621c59d8d6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:38.465949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:38.468082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.468531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.470619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.473159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.493591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3f060-a51f-4b85-8867-1da61e7038b2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:38.498973 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:38.499780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.500071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.500383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] PATH_INFO: `/v3/projects/c3e27f6e9d9045fbae8eeb5b8dd85d9c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.502463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e27f6e9d9045fbae8eeb5b8dd85d9c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:38.504528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:38.560117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:38.561583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50d183-2ae8-4065-b2f5-822effe2216e service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:38.571528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e27f6e9d9045fbae8eeb5b8dd85d9c', 'request_id': 'req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:38.572300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 998/1976] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:38.578652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e27f6e9d9045fbae8eeb5b8dd85d9c', 'request_id': 'req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:38.587066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e27f6e9d9045fbae8eeb5b8dd85d9c', 'request_id': 'req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:38.606356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] The token cache is being invalidate because project c3e27f6e9d9045fbae8eeb5b8dd85d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:38.607046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:38.608166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-39eefbcb-f5f8-4c3f-8a97-f5843c72b1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:38.614358 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 979/1977] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:38 2022] DELETE /identity/v3/projects/c3e27f6e9d9045fbae8eeb5b8dd85d9c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:38.616228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.616705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.617950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.643003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:38.644176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.721999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 999/1978] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:38.729715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.730493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.730879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.757302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:38.758732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:38.767336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 980/1979] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:38.773259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:38.781013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:38.786230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45c3b-c2b6-45ed-99ed-d2f864837853 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:38.793556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:38.795174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.795492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.795770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.796915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:38.797216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:38.798866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:38.814546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1000/1980] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:31:38.820273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:38.831340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea517d-c6b2-4e41-a855-a1eaa536d667 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:38.838515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:38.839718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:38.840440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:38.841014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:38.841986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:38.844265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:38.893308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 981/1981] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:38 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:38.893680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:38.908943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa071d-de69-4f25-9743-3a5dbcd7cb6e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.026205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.028708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.029746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.030717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.043372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.043372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.075363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1001/1982] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:38 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:39.079392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.093399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffdc8a-716d-4641-a131-48f319300891 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.100656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.101312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.101778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.102199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.103031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.104850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.122367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.128554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 982/1983] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:39.136907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6f819-5789-4ca4-a1db-900355bd9f27 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.143458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133c3fd-c7e4-44b9-bf8f-e2b6cddba24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.144289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133c3fd-c7e4-44b9-bf8f-e2b6cddba24d admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.144655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133c3fd-c7e4-44b9-bf8f-e2b6cddba24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.144986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133c3fd-c7e4-44b9-bf8f-e2b6cddba24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.145861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133c3fd-c7e4-44b9-bf8f-e2b6cddba24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.152296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8133c3fd-c7e4-44b9-bf8f-e2b6cddba24d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.166998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1002/1984] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:39.171344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.182566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5267d-b949-457e-a3f1-2deaa224aca9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.195620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.196478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.196943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.197370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] PATH_INFO: `/v3/projects/eb8e5db7d42b4c76ada7512c611b2eab/users/7f43ef46774844f1ab348c53ad1a5362/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.218845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8e5db7d42b4c76ada7512c611b2eab, user_id=7f43ef46774844f1ab348c53ad1a5362, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.221649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.239998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 983/1985] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:39 2022] PUT /identity/v3/projects/eb8e5db7d42b4c76ada7512c611b2eab/users/7f43ef46774844f1ab348c53ad1a5362/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:39.251772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.252055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.252325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.285325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] MFA Rules not processed for user `7f43ef46774844f1ab348c53ad1a5362`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:39.286569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:39.369396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1003/1986] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:39.370540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.417711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:39.433867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a4fad1-b218-4759-8155-1584aa45926d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.438482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.439129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.439414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.439684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.440299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.442267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.471614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 984/1987] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:39 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:39.475664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.486739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5acc6-a82e-43bf-a32a-8c313332600c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.549234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.549851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.550133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.550408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.551249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.553697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.561497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:39.576255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1004/1988] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:39 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:39.580229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.590530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdb32-c488-46ea-a910-c7a6e6005375 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.595650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.596280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.596574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.596833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.597471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.599115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.609567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 985/1989] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:31:39.613419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.624265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5a54f-4556-429c-960c-3f589f47139f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.628825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e24e7-7a01-4ae9-b500-9831bc9cbddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.629448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e24e7-7a01-4ae9-b500-9831bc9cbddb admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.629727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e24e7-7a01-4ae9-b500-9831bc9cbddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.629995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e24e7-7a01-4ae9-b500-9831bc9cbddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.631199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e24e7-7a01-4ae9-b500-9831bc9cbddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.632776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e24e7-7a01-4ae9-b500-9831bc9cbddb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.642596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1005/1990] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:39.646778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.656866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663cd5b-d12e-4499-a0ea-4514dc829903 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.661439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.662096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.662377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.662686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] PATH_INFO: `/v3/projects/e5aa8733b9ed4cd3ba729afb9469803c/users/8c216a706a284a01a3304f118821eae2/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.678093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5aa8733b9ed4cd3ba729afb9469803c, user_id=8c216a706a284a01a3304f118821eae2, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.680780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.695667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 986/1991] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:39 2022] PUT /identity/v3/projects/e5aa8733b9ed4cd3ba729afb9469803c/users/8c216a706a284a01a3304f118821eae2/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:39.706070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.706356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.706640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.734996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] MFA Rules not processed for user `8c216a706a284a01a3304f118821eae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:39.736266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:39.806724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1006/1992] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:39.811495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.878088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c12de0-b151-46ba-bc54-e8269b1efe90 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.883367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.883969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.884247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.884519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.885171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.887072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:39.916071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 987/1993] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:39 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:39.920592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:39.931045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b53e16-4d75-4422-9059-c635ec291174 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:39.985724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:39.986327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:39.986609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:39.986879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:39.987546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:39.989460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:40.010576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1007/1994] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:39 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:40.014834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:40.026207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262988d-bc8e-4227-ba90-4197b599596e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:40.030847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:40.031473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:40.031755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:40.032026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:40.032668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:40.034267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:40.045060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 988/1995] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:40.049289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:40.059333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07baca7-d1ec-4756-975f-08b6ea7086a9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:40.064259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:40.064909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:40.065187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:40.065457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] PATH_INFO: `/v3/projects/043e6c04d3da4311b23152c18c844fdd/users/389690f2db524a45be129def23d12cf1/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:40.080748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=043e6c04d3da4311b23152c18c844fdd, user_id=389690f2db524a45be129def23d12cf1, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:40.083398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:40.097569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1008/1996] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:40 2022] PUT /identity/v3/projects/043e6c04d3da4311b23152c18c844fdd/users/389690f2db524a45be129def23d12cf1/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:40.101505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:40.111531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833a31e-316a-4de7-9320-a9df73701c1a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:40.169203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e2a90-10f2-4599-aff7-25d26e4d96b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:40.170071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2a90-10f2-4599-aff7-25d26e4d96b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:40.170353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2a90-10f2-4599-aff7-25d26e4d96b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:40.170676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e2a90-10f2-4599-aff7-25d26e4d96b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:40.171296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2a90-10f2-4599-aff7-25d26e4d96b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:40.172890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65e2a90-10f2-4599-aff7-25d26e4d96b0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:40.183553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 989/1997] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:40.187689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:40.197845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0a55-ae4a-4d67-84d1-ff093137f47a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:40.202483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:40.203154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:40.203439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:40.203716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] PATH_INFO: `/v3/projects/043e6c04d3da4311b23152c18c844fdd/users/389690f2db524a45be129def23d12cf1/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:40.205625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=043e6c04d3da4311b23152c18c844fdd, user_id=389690f2db524a45be129def23d12cf1, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:40.208304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:40.218943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1009/1998] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:40 2022] PUT /identity/v3/projects/043e6c04d3da4311b23152c18c844fdd/users/389690f2db524a45be129def23d12cf1/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:40.230278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:40.230566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:40.230863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:40.246969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:40.270396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] MFA Rules not processed for user `389690f2db524a45be129def23d12cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:40.272346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:40.353124 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 990/1999] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:40.365957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:40.368954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:40.394237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:40.451272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4053e-bfa7-4327-a5af-886a5cd28ab8 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:40.453914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:40.454536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:40.454816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:40.455109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:40.527271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:40.529364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:40.540672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1010/2000] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:42.592789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:42.594213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:42.607387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1eea0-f018-4e38-a4c4-67b39dbf413e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:42.611207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:42.611806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:42.612081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:42.612354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:42.614057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:42.678410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:42.679897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:42.689550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 991/2001] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:48.474777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:48.484330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087afd37-faa4-4a54-bae0-db320c845b3e service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:48.488168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:48.488988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:48.489272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:48.489550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:48.551877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:48.553355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:48.562918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1011/2002] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:49.204704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:49.214820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e22552-d8a0-441f-afe4-4fb94fd9ede4 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:49.220018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fb2dc-b122-4c2f-8ad6-d674d59ab896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:49.220748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fb2dc-b122-4c2f-8ad6-d674d59ab896 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:49.221035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fb2dc-b122-4c2f-8ad6-d674d59ab896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:49.221324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fb2dc-b122-4c2f-8ad6-d674d59ab896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:49.247622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fb2dc-b122-4c2f-8ad6-d674d59ab896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:49.257247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fb2dc-b122-4c2f-8ad6-d674d59ab896 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:49.282422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 992/2003] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:49.778873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:49.781259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:49.902920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cfcad-d2be-45de-8c66-eaca7695af1c service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:49.928180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:49.928755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:49.929044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:49.929523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] PATH_INFO: `/v3/users/1bd41e80e37a40b98f6a01b947692181` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:49.931468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd41e80e37a40b98f6a01b947692181)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:49.933450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:49.954576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:50.011280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd41e80e37a40b98f6a01b947692181'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:50.019663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd41e80e37a40b98f6a01b947692181'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:50.020344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd41e80e37a40b98f6a01b947692181'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:50.028876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd41e80e37a40b98f6a01b947692181'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:50.042105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd41e80e37a40b98f6a01b947692181'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:50.048894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1012/2004] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:49 2022] DELETE /identity/v3/users/1bd41e80e37a40b98f6a01b947692181 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:50.053573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:50.054107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:50.055184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:50.075297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:50.076621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:50.148266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 993/2005] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:50.213642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:50.216246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:50.293273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd8126-72e3-466b-9ac6-64456de7fd3a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:50.297205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:50.297827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:50.298161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:50.298436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:50.311279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:50.314218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:50.329062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1013/2006] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:50.456387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:50.457974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:50.476789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:50.542612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a836e6c-0770-4aa3-92e6-4c4e32083f07 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:50.547415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:50.548380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:50.550014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:50.550397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:50.556032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:50.690928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:50.692718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6384e66-642e-457b-8de1-66e71eba9f62 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:50.707881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 994/2007] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:51.184402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:51.185898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:51.206436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a979c-98bd-4107-94f8-851bb960ec5c service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:51.223014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:51.223790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.225010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.225458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] PATH_INFO: `/v3/projects/caca9db1e0174372966bd1b6b8548619` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.231665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] RBAC: Authorizing `identity:delete_project(project_id=caca9db1e0174372966bd1b6b8548619)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:51.233591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:51.340852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caca9db1e0174372966bd1b6b8548619', 'request_id': 'req-5055c270-4d0a-40e0-8140-7c82e78da34a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:51.349506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caca9db1e0174372966bd1b6b8548619', 'request_id': 'req-5055c270-4d0a-40e0-8140-7c82e78da34a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:51.357819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caca9db1e0174372966bd1b6b8548619', 'request_id': 'req-5055c270-4d0a-40e0-8140-7c82e78da34a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:51.394692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] The token cache is being invalidate because project caca9db1e0174372966bd1b6b8548619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:51.394692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:51.394692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5055c270-4d0a-40e0-8140-7c82e78da34a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:51.394692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1014/2008] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:51 2022] DELETE /identity/v3/projects/caca9db1e0174372966bd1b6b8548619 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:51.398328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.398619 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.398898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.436161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:51.437397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:51.543164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.543584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.543989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.553278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 995/2009] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:51.574242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:51.576183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:51.587447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1015/2010] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:51.592451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:51.604674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd65cb-7650-4e4a-93d4-490e87be4b41 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:51.609639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:51.610561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.611045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.611552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.612824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:51.613340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:51.615202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:51.634413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:51.634617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 996/2011] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:51.645248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a813f0-3113-4632-8953-7541daa48cf4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:51.650634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:51.651597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.652075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.652522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.653352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:51.655940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:51.667028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:51.706103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1016/2012] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:51 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:51.711151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:51.731858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bba1d4-5d3f-4535-ac05-a62e1df5777a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:51.837387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:51.861360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:51.862014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.862298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.862573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.863402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:51.865552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:51.914726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 997/2013] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:51 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:31:51.915281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:51.934516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39180a-71cc-4cdf-a3e4-b18f1953a9cd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:51.943305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:51.943931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:51.944236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:51.944510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:51.945776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:51.947436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:51.961770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1017/2014] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:51.965593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:51.980017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cc207-c06b-49fb-ad8e-b69353a909bb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:52.004831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22fb3-8040-4a10-9fb1-b4258a0d65ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:52.005530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22fb3-8040-4a10-9fb1-b4258a0d65ae admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:52.005867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22fb3-8040-4a10-9fb1-b4258a0d65ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:52.006236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22fb3-8040-4a10-9fb1-b4258a0d65ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:52.007185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f22fb3-8040-4a10-9fb1-b4258a0d65ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:52.009589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f22fb3-8040-4a10-9fb1-b4258a0d65ae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:52.023972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 998/2015] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:52.030398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:52.051891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b82c7-ec7b-48e9-9b19-85cc60ca7aef admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:52.064070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:52.064893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:52.066243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:52.066604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] PATH_INFO: `/v3/projects/8aac3d3b720c45e8817bda1fe4d2d157/users/2495d3c63dc44049872afb908ca6133b/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:52.134057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aac3d3b720c45e8817bda1fe4d2d157, user_id=2495d3c63dc44049872afb908ca6133b, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:52.139671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:52.172930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1018/2016] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:52 2022] PUT /identity/v3/projects/8aac3d3b720c45e8817bda1fe4d2d157/users/2495d3c63dc44049872afb908ca6133b/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:52.178716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:52.179876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:52.180959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:52.230116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:52.232788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:52.328790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 999/2017] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:52.344733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:52.348076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:52.444240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28304ef4-5d7e-4171-8f36-dab23d575c40 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:52.451627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7df0fc-9807-4021-9408-e14faae7013a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:52.452672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df0fc-9807-4021-9408-e14faae7013a service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:52.453234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df0fc-9807-4021-9408-e14faae7013a service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:52.453706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df0fc-9807-4021-9408-e14faae7013a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:52.472657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7df0fc-9807-4021-9408-e14faae7013a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:52.474444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7df0fc-9807-4021-9408-e14faae7013a service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:52.496932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1019/2018] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:56.607466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:56.609390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:56.621928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a0388-e0c5-47d9-858c-c19f92fbe846 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:56.629178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:56.630189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:56.630622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:56.631045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] PATH_INFO: `/v3/users/7f43ef46774844f1ab348c53ad1a5362` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:56.632426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f43ef46774844f1ab348c53ad1a5362)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:56.635079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:56.707175 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f43ef46774844f1ab348c53ad1a5362'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:56.717376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f43ef46774844f1ab348c53ad1a5362'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:56.731213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f43ef46774844f1ab348c53ad1a5362'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:56.731931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f43ef46774844f1ab348c53ad1a5362'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:56.748965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f43ef46774844f1ab348c53ad1a5362'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:56.757954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1000/2019] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:56 2022] DELETE /identity/v3/users/7f43ef46774844f1ab348c53ad1a5362 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:56.762837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:56.763683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:56.764142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:56.794138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:56.796448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:56.866982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1020/2020] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:56.877305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:56.879479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:56.935033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:56.951998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894761ec-d0b3-4804-9002-d5eed5ff3281 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:56.956561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:56.957894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:56.958081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:56.958476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:56.969477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:56.970926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:56.981251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1001/2021] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:31:57.284258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:57.285775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:57.307473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f5981-9761-4143-830c-0e613e0eda31 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:57.315417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:57.316318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:57.316593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:57.317013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] PATH_INFO: `/v3/projects/eb8e5db7d42b4c76ada7512c611b2eab` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:57.318592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8e5db7d42b4c76ada7512c611b2eab)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:57.322076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:57.382008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8e5db7d42b4c76ada7512c611b2eab', 'request_id': 'req-9bfeb049-e655-48d6-a8a2-459b588cdb34'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.385049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:57.391961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8e5db7d42b4c76ada7512c611b2eab', 'request_id': 'req-9bfeb049-e655-48d6-a8a2-459b588cdb34'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.400827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8e5db7d42b4c76ada7512c611b2eab', 'request_id': 'req-9bfeb049-e655-48d6-a8a2-459b588cdb34'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.419469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] The token cache is being invalidate because project eb8e5db7d42b4c76ada7512c611b2eab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:57.419997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.420645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.422777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1021/2022] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:57 2022] DELETE /identity/v3/projects/eb8e5db7d42b4c76ada7512c611b2eab => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:57.426663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:57.440500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f82fb4-c8ac-4de5-8478-11f3eaae3e91 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:57.531296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:57.538359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:57.538676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:57.538978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] PATH_INFO: `/v3/users/8c216a706a284a01a3304f118821eae2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:57.542186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c216a706a284a01a3304f118821eae2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:57.544504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:57.639604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c216a706a284a01a3304f118821eae2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.656531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c216a706a284a01a3304f118821eae2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.680839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c216a706a284a01a3304f118821eae2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.681463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c216a706a284a01a3304f118821eae2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.698684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c216a706a284a01a3304f118821eae2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:57.705706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1002/2023] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:57 2022] DELETE /identity/v3/users/8c216a706a284a01a3304f118821eae2 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:58.071414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.072911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:58.090054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeb049-e655-48d6-a8a2-459b588cdb34 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.176155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.179765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.180193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.180657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] PATH_INFO: `/v3/projects/e5aa8733b9ed4cd3ba729afb9469803c` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.182326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5aa8733b9ed4cd3ba729afb9469803c)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.184427 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:58.249773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5aa8733b9ed4cd3ba729afb9469803c', 'request_id': 'req-9ce73eeb-9f62-4599-881e-f0e6283729ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.260794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5aa8733b9ed4cd3ba729afb9469803c', 'request_id': 'req-9ce73eeb-9f62-4599-881e-f0e6283729ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.271448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5aa8733b9ed4cd3ba729afb9469803c', 'request_id': 'req-9ce73eeb-9f62-4599-881e-f0e6283729ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.307504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] The token cache is being invalidate because project e5aa8733b9ed4cd3ba729afb9469803c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:58.308057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.309073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-9ce73eeb-9f62-4599-881e-f0e6283729ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.313863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1022/2024] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:58 2022] DELETE /identity/v3/projects/e5aa8733b9ed4cd3ba729afb9469803c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:58.314962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.319185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:58.335064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c071f0-139a-4679-ae49-889b003e0bb4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.397453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.397739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.398016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.429845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.431195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.431607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.431982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] PATH_INFO: `/v3/users/389690f2db524a45be129def23d12cf1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.433288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389690f2db524a45be129def23d12cf1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.435424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:58.435735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] MFA Rules not processed for user `2495d3c63dc44049872afb908ca6133b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:58.437272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:58.446644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:58.509807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389690f2db524a45be129def23d12cf1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.527293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389690f2db524a45be129def23d12cf1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.534156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:58.539844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1023/2025] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:58.542470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389690f2db524a45be129def23d12cf1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.543629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389690f2db524a45be129def23d12cf1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.545038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.546822 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:58.562574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389690f2db524a45be129def23d12cf1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:58.574864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1003/2026] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:31:58 2022] DELETE /identity/v3/users/389690f2db524a45be129def23d12cf1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:58.575916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39a8c-7d56-4623-8492-a161b6d40736 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.689810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.690720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.691066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.691373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.692657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.694907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:58.752913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1024/2027] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:58 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:58.761631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.763374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:58.776779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214487f-b929-4713-b353-6354ca6f7875 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.853161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.853913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.854302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.854677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.855530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.857548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:58.891134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1004/2028] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:58 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:58.894014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.905053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611d442-417c-4967-a86d-a30180585e6c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.909896 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.910658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.910935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.911314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.911968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.913617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:58.926344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1025/2029] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:58.931195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.946343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551df3c6-99b1-4eec-a349-799794cc7bbe admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.958463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.959118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.959418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.959562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:58.959707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] PATH_INFO: `/v3/projects/c21408fcc6b64efa8d742f6bdf415ef9/users/16e72667b9ff4e86858c9fd057b47de3/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.962847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:58.984835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21408fcc6b64efa8d742f6bdf415ef9, user_id=16e72667b9ff4e86858c9fd057b47de3, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.985383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c02fb-1bc0-4ea5-abf7-d9b6638b7313 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:58.987674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:58.991501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:58.993183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:58.993362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:58.994076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] PATH_INFO: `/v3/projects/043e6c04d3da4311b23152c18c844fdd` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:58.995533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=043e6c04d3da4311b23152c18c844fdd)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:58.998174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.013306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1005/2030] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:58 2022] PUT /identity/v3/projects/c21408fcc6b64efa8d742f6bdf415ef9/users/16e72667b9ff4e86858c9fd057b47de3/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:59.014710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.056133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '043e6c04d3da4311b23152c18c844fdd', 'request_id': 'req-44098154-231c-451e-b4e4-982e7faba8fa'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:59.062331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '043e6c04d3da4311b23152c18c844fdd', 'request_id': 'req-44098154-231c-451e-b4e4-982e7faba8fa'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:59.070469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '043e6c04d3da4311b23152c18c844fdd', 'request_id': 'req-44098154-231c-451e-b4e4-982e7faba8fa'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:59.077994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:59.084773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] The token cache is being invalidate because project 043e6c04d3da4311b23152c18c844fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:31:59.085257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:59.086214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-44098154-231c-451e-b4e4-982e7faba8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:31:59.090034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1026/2031] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:31:58 2022] DELETE /identity/v3/projects/043e6c04d3da4311b23152c18c844fdd => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:59.094890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01874df1-5c97-44bf-b108-4b076064573e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.095284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01874df1-5c97-44bf-b108-4b076064573e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.095420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4bba3-08d3-4a3a-a2d2-95ac223c96f3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.095830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01874df1-5c97-44bf-b108-4b076064573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.118205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-01874df1-5c97-44bf-b108-4b076064573e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:59.119488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-01874df1-5c97-44bf-b108-4b076064573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:59.167068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.167793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.168543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.168852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.169852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.171603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.178733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:59.184308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1006/2032] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:31:59.188333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.190208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:59.191240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1027/2033] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:59.197408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.197985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.198277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.202628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047bde0-9fdf-47c3-887f-594690d4d83e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.208075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.208730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.209008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.209284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] PATH_INFO: `/v3/projects/c21408fcc6b64efa8d742f6bdf415ef9/users/16e72667b9ff4e86858c9fd057b47de3/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.211218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21408fcc6b64efa8d742f6bdf415ef9, user_id=16e72667b9ff4e86858c9fd057b47de3, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.214378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.219223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:59.220418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:59.224871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1007/2034] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:59 2022] PUT /identity/v3/projects/c21408fcc6b64efa8d742f6bdf415ef9/users/16e72667b9ff4e86858c9fd057b47de3/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:31:59.286600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1028/2035] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:31:59.291832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.306620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef34f6c-22a2-46a2-9c2c-f275154b5fdc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.314864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.319216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.319587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.319972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.322469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:31:59.322793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.325084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.350270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.357035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1008/2036] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:31:59.360847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96916bd2-558e-4b7b-a4a1-39ab8ba2d833 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.365922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.366540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.366825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.367530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.368232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.370268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.404477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1029/2037] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:31:59 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:31:59.409479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.424872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348b7e-2445-49be-b480-88b5b220f2cc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.500184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.500888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.501168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.501445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.502095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.504233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.535259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1009/2038] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:31:59 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:31:59.539617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.551733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c4416-2829-4578-893a-d8385cd8eb34 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.556722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.557252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.557533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.557808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.558464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.560144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.576109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1030/2039] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:59.580216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.594306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b23cb-da2b-4484-9afb-262bf68f8f4b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.601741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad09f3-c9ae-42aa-bb4e-9a1731d7791d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.603575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad09f3-c9ae-42aa-bb4e-9a1731d7791d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.603921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad09f3-c9ae-42aa-bb4e-9a1731d7791d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.604246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad09f3-c9ae-42aa-bb4e-9a1731d7791d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.605178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad09f3-c9ae-42aa-bb4e-9a1731d7791d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.607611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad09f3-c9ae-42aa-bb4e-9a1731d7791d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.622563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1010/2040] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:31:59.632040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.647675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b038b6-a86d-4959-927e-42d41d627762 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.655308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.656098 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.656382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.656820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] PATH_INFO: `/v3/projects/d6dae806b3c14f3fa1d2a491214de9f8/users/c29b21ee2cd341169a2c8263393c5db6/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.683635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6dae806b3c14f3fa1d2a491214de9f8, user_id=c29b21ee2cd341169a2c8263393c5db6, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:31:59.693622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:31:59.728382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b47c3-e84b-4feb-ad7c-e1262fea6ad3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.730488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b47c3-e84b-4feb-ad7c-e1262fea6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.731339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b47c3-e84b-4feb-ad7c-e1262fea6ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:31:59.735170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1031/2041] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:31:59 2022] PUT /identity/v3/projects/d6dae806b3c14f3fa1d2a491214de9f8/users/c29b21ee2cd341169a2c8263393c5db6/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:31:59.767841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-117b47c3-e84b-4feb-ad7c-e1262fea6ad3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:31:59.769604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-117b47c3-e84b-4feb-ad7c-e1262fea6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:59.828548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:31:59.852821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1011/2042] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:31:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:31:59.887518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:31:59.890706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:31:59.992096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e87707-f04b-411b-b88b-67b3a36ccfb9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:31:59.996449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:31:59.997358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:31:59.997478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:31:59.997801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:00.009610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:00.012362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:00.027552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1032/2043] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:31:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:00.938474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:00.939188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:00.939188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:00.973728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] MFA Rules not processed for user `c29b21ee2cd341169a2c8263393c5db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:00.975538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:01.001389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:01.005470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:01.042504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23020e5b-062d-434a-9b82-77a8a746117e service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:01.052645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:01.053699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:01.054232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:01.054536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] PATH_INFO: `/v3/users/dfff7de270c140b2800a8b53c1157fc3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:01.057545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfff7de270c140b2800a8b53c1157fc3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:01.060176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:01.080388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1012/2044] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:01.094587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:01.098117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:01.129820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:01.142007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfff7de270c140b2800a8b53c1157fc3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:01.155570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfff7de270c140b2800a8b53c1157fc3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:01.157477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfff7de270c140b2800a8b53c1157fc3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:01.177766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfff7de270c140b2800a8b53c1157fc3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:01.198501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1641d3b1-52de-4235-9733-dcc2dcbe1148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfff7de270c140b2800a8b53c1157fc3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:01.203329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda86c1-2aa9-4033-b4bc-11fdaeab1685 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:01.207156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1033/2045] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:00 2022] DELETE /identity/v3/users/dfff7de270c140b2800a8b53c1157fc3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:32:01.209967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:01.210668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:01.211261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:01.212322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:01.215018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:01.215592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:01.216069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:01.216647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:01.231329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:01.233657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:01.242154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:01.244618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:01.260190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1013/2046] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:01.345314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1034/2047] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:01.358123 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:01.359677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:01.380385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf19934-40e1-449f-bf3f-06294b6a765e service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:01.458224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:01.459174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:01.460475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:01.461144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:01.473743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:01.476120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:01.500794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1014/2048] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:02.078741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:02.080459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:02.100143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3258e5-8da9-4fad-a7ee-58fd05d4d45d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:02.105897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:02.106838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:02.107325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:02.107678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] PATH_INFO: `/v3/projects/2d566366c64641a69e791f7208532828` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:02.109499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d566366c64641a69e791f7208532828)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:02.110925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:02.186558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d566366c64641a69e791f7208532828', 'request_id': 'req-d46ebf11-f85a-4f16-9c52-c681e2474aee'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.194323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d566366c64641a69e791f7208532828', 'request_id': 'req-d46ebf11-f85a-4f16-9c52-c681e2474aee'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.201441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d566366c64641a69e791f7208532828', 'request_id': 'req-d46ebf11-f85a-4f16-9c52-c681e2474aee'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.218611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] The token cache is being invalidate because project 2d566366c64641a69e791f7208532828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:02.218985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.219798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d46ebf11-f85a-4f16-9c52-c681e2474aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.221622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1035/2049] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:02 2022] DELETE /identity/v3/projects/2d566366c64641a69e791f7208532828 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:02.225422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:02.236961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51ff0-e0c3-40e1-b85d-2e3869fc6d53 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:02.254008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:02.297938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:02.298815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:02.299066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:02.299349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] PATH_INFO: `/v3/users/8faac3c22a3a4493bae765fc0feee140` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:02.300709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8faac3c22a3a4493bae765fc0feee140)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:02.304604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:02.384499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8faac3c22a3a4493bae765fc0feee140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.391904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8faac3c22a3a4493bae765fc0feee140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.403678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8faac3c22a3a4493bae765fc0feee140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.404364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8faac3c22a3a4493bae765fc0feee140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.418683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8faac3c22a3a4493bae765fc0feee140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:02.428213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1015/2050] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:02 2022] DELETE /identity/v3/users/8faac3c22a3a4493bae765fc0feee140 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:02.972393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad0b1a-9934-4940-b8f0-1c3a486815ce None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:02.972393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad0b1a-9934-4940-b8f0-1c3a486815ce None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:02.972393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad0b1a-9934-4940-b8f0-1c3a486815ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:03.038983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-10ad0b1a-9934-4940-b8f0-1c3a486815ce None None] MFA Rules not processed for user `c29b21ee2cd341169a2c8263393c5db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:03.040684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-10ad0b1a-9934-4940-b8f0-1c3a486815ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:03.126914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:03.176162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1036/2051] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:03.246111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:03.248522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:03.388598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db074af5-a4b6-482a-83fa-25177dba293e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:03.399470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:03.400421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:03.400881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:03.401312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] PATH_INFO: `/v3/projects/dbaabd61abfb42289e1ac64a793298d7` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:03.402764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbaabd61abfb42289e1ac64a793298d7)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:03.405572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:03.467965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:03.478837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbaabd61abfb42289e1ac64a793298d7', 'request_id': 'req-3626a756-a627-41a7-b34a-1b9f2e05c0dc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:03.486875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbaabd61abfb42289e1ac64a793298d7', 'request_id': 'req-3626a756-a627-41a7-b34a-1b9f2e05c0dc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:03.496821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbaabd61abfb42289e1ac64a793298d7', 'request_id': 'req-3626a756-a627-41a7-b34a-1b9f2e05c0dc'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:03.518658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] The token cache is being invalidate because project dbaabd61abfb42289e1ac64a793298d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:03.519037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:03.519798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3626a756-a627-41a7-b34a-1b9f2e05c0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:03.523272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1016/2052] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:03 2022] DELETE /identity/v3/projects/dbaabd61abfb42289e1ac64a793298d7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:03.528193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:03.528496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:03.528779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:03.570721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:03.571957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:03.740046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1037/2053] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:03.740887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:03.741385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:03.741673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:03.788671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:03.789990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:03.814623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:03.825527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1017/2054] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:03.835170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642937-96ae-454e-9168-9c18f1b25986 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:03.840692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:03.841566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:03.841802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:03.842213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:03.848363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:03.848682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:03.850383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:03.892338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:03.898988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1038/2055] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:03.906078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45fcb-65e2-47db-9766-4315f7a12c10 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:03.950523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:03.951361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:03.955783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:03.956096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:03.957105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:03.959361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:04.019523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1018/2056] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:03 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:04.021665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:04.037286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8a0f3-f573-4a70-93da-0563a98f2c2a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:04.151195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:04.155829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:04.155829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:04.156091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:04.160276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:04.175246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:04.240200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1039/2057] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:04 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:04.241719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:04.258479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a6205-a80e-40ce-9973-b5d2f16790e6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:04.276904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734335-09b1-4997-a966-54a77746632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:04.277553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734335-09b1-4997-a966-54a77746632d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:04.277831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734335-09b1-4997-a966-54a77746632d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:04.278099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734335-09b1-4997-a966-54a77746632d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:04.278755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734335-09b1-4997-a966-54a77746632d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:04.281425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734335-09b1-4997-a966-54a77746632d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:04.314565 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1019/2058] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:04.316887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:04.329785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbba24c-ea15-44c3-ade6-87547d0594e7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:04.335561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627ec8e-c880-4f05-bd92-e0a4e95c3291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:04.336731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627ec8e-c880-4f05-bd92-e0a4e95c3291 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:04.337014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627ec8e-c880-4f05-bd92-e0a4e95c3291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:04.337299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627ec8e-c880-4f05-bd92-e0a4e95c3291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:04.337950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627ec8e-c880-4f05-bd92-e0a4e95c3291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:04.339691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627ec8e-c880-4f05-bd92-e0a4e95c3291 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:04.357404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1040/2059] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:04.362379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734335-09b1-4997-a966-54a77746632d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:04.376373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734335-09b1-4997-a966-54a77746632d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:04.393763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:04.394639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:04.395075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:04.397384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] PATH_INFO: `/v3/projects/a254b84d9a7a43be9d16212e4a7ef264/users/7391777d67934b0da0e5273dac73d061/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:04.447139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a254b84d9a7a43be9d16212e4a7ef264, user_id=7391777d67934b0da0e5273dac73d061, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:04.455263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:04.502935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1020/2060] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:04 2022] PUT /identity/v3/projects/a254b84d9a7a43be9d16212e4a7ef264/users/7391777d67934b0da0e5273dac73d061/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:04.515233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:04.515522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:04.515801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:04.613771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] MFA Rules not processed for user `7391777d67934b0da0e5273dac73d061`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:04.615038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:04.754185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:04.837731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1041/2061] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:04.842523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.007672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca69f74-8bb0-47b5-9adc-0f9966b07757 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.019874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.021277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.021718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.022134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.022938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:05.025564 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:05.072778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.073433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1021/2062] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:04 2022] POST /identity/v3/projects => generated 371 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:05.089480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758893d-8a35-4cbb-a1c2-5e579338897b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.245323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.245977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.246327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.246469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.247199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:05.250357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:05.299713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1042/2063] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:05 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:05.304304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.330983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261733c-9f12-4095-b327-a59d00036cc6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.345156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036fb9aa-4e2d-4e3e-be1a-aef477bba352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.345796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fb9aa-4e2d-4e3e-be1a-aef477bba352 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.346080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fb9aa-4e2d-4e3e-be1a-aef477bba352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.346357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036fb9aa-4e2d-4e3e-be1a-aef477bba352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.347015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fb9aa-4e2d-4e3e-be1a-aef477bba352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:05.349022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036fb9aa-4e2d-4e3e-be1a-aef477bba352 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:05.371348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1022/2064] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:05.375949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.396267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada0e03-4a58-4e8f-812c-a5b689023aec admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.415529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.416220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.416493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.416765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] PATH_INFO: `/v3/projects/c4b484ab5466406a98e6e56ab4f8087f/users/2031f3acc29542cc93b19e6fde390ff3/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.445938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.446256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.446528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.507959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b484ab5466406a98e6e56ab4f8087f, user_id=2031f3acc29542cc93b19e6fde390ff3, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:05.515050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:05.529363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] MFA Rules not processed for user `16e72667b9ff4e86858c9fd057b47de3`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:05.530801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:05.546413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1043/2065] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:05 2022] PUT /identity/v3/projects/c4b484ab5466406a98e6e56ab4f8087f/users/2031f3acc29542cc93b19e6fde390ff3/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:05.550173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.611928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:05.695076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92829e74-250f-4232-963c-f3a2ac31c2e7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.710343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.713071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.713424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.713704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.714711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1023/2066] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:05.714838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:05.716657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:05.737118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1044/2067] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:05 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:05.740644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.743138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:05.747363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:05.780309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79324ebd-cd4c-4a9b-9c3c-7fe7f1f8816c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.793195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b0603-f1a4-4d4b-aaff-2b1a17df88db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.793897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b0603-f1a4-4d4b-aaff-2b1a17df88db admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.794410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b0603-f1a4-4d4b-aaff-2b1a17df88db admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.794681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b0603-f1a4-4d4b-aaff-2b1a17df88db admin admin] PATH_INFO: `/v3/projects/c4b484ab5466406a98e6e56ab4f8087f/users/2031f3acc29542cc93b19e6fde390ff3/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.799028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b0603-f1a4-4d4b-aaff-2b1a17df88db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4b484ab5466406a98e6e56ab4f8087f, user_id=2031f3acc29542cc93b19e6fde390ff3, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:05.802365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b0603-f1a4-4d4b-aaff-2b1a17df88db admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:05.828748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1045/2068] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:05 2022] PUT /identity/v3/projects/c4b484ab5466406a98e6e56ab4f8087f/users/2031f3acc29542cc93b19e6fde390ff3/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:05.840169 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.842508 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.842639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.897717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972203a4-8195-45f1-8810-616f066f11b6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:05.902046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] MFA Rules not processed for user `2031f3acc29542cc93b19e6fde390ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:05.903304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:05.928275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:05.979882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:05.979882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:05.980198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:05.980284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:05.982103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:05.993784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1046/2069] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:32:06.006314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:06.008398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:06.062400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:06.064575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:06.069935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:06.076401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3f85c-d0be-49b5-a5e6-22ce50a218f6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:06.078312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1024/2070] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:32:06.080424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c918d2-621b-469f-a363-2652ac9d26ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:06.081040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c918d2-621b-469f-a363-2652ac9d26ea service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:06.081320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c918d2-621b-469f-a363-2652ac9d26ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:06.081593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c918d2-621b-469f-a363-2652ac9d26ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:06.170995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c918d2-621b-469f-a363-2652ac9d26ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:06.173466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c918d2-621b-469f-a363-2652ac9d26ea service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:06.198774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1047/2071] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:09.211829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:09.340415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430fc7b-8fd4-4780-a1a6-51f9c0bfaee1 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:09.345548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:09.346196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:09.346482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:09.346755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] PATH_INFO: `/v3/users/7391777d67934b0da0e5273dac73d061` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:09.347935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7391777d67934b0da0e5273dac73d061)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:09.349891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:09.423333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7391777d67934b0da0e5273dac73d061'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:09.434106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7391777d67934b0da0e5273dac73d061'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:09.452013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7391777d67934b0da0e5273dac73d061'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:09.452782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7391777d67934b0da0e5273dac73d061'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:09.469425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7391777d67934b0da0e5273dac73d061'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:09.476499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1025/2072] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:09 2022] DELETE /identity/v3/users/7391777d67934b0da0e5273dac73d061 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:09.480804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:09.481683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:09.482051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:09.513309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:09.514687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:09.579496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1048/2073] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:09.590651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:09.592212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:09.657092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff303fb2-7fae-4f13-82a3-6bb30aa5209a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:09.661382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:09.662021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:09.662426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:09.662717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:09.675011 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:09.676782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:09.688910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1026/2074] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:32:09.992059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:09.994396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:10.007772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546e5c9-396b-427b-8938-ef1a4d5ca6cb None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:10.013071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:10.014026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.014398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.014771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] PATH_INFO: `/v3/projects/a254b84d9a7a43be9d16212e4a7ef264` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:10.016097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a254b84d9a7a43be9d16212e4a7ef264)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:10.018287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:10.021464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:10.079991 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a254b84d9a7a43be9d16212e4a7ef264', 'request_id': 'req-1f4305ad-0be0-43d5-8d62-93740d50ce64'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.085826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a254b84d9a7a43be9d16212e4a7ef264', 'request_id': 'req-1f4305ad-0be0-43d5-8d62-93740d50ce64'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.092737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a254b84d9a7a43be9d16212e4a7ef264', 'request_id': 'req-1f4305ad-0be0-43d5-8d62-93740d50ce64'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.107290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] The token cache is being invalidate because project a254b84d9a7a43be9d16212e4a7ef264 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:10.107606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.108243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.109808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1049/2075] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:09 2022] DELETE /identity/v3/projects/a254b84d9a7a43be9d16212e4a7ef264 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:10.113803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:10.124443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f169f-88e4-4292-b3fc-071b1d2f9f76 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:10.144286 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:10.192030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:10.192664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.192939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.193209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] PATH_INFO: `/v3/users/2031f3acc29542cc93b19e6fde390ff3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:10.194350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2031f3acc29542cc93b19e6fde390ff3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:10.196368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:10.254166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2031f3acc29542cc93b19e6fde390ff3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.269721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2031f3acc29542cc93b19e6fde390ff3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.272399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2031f3acc29542cc93b19e6fde390ff3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.273338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2031f3acc29542cc93b19e6fde390ff3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.298059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2031f3acc29542cc93b19e6fde390ff3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.305882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1027/2076] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:10 2022] DELETE /identity/v3/users/2031f3acc29542cc93b19e6fde390ff3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:10.525334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:10.527471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:10.549007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4305ad-0be0-43d5-8d62-93740d50ce64 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:10.654473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:10.655214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.655718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.656137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] PATH_INFO: `/v3/users/6dd452efd6a8471fb6e58171882f3067` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:10.657564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd452efd6a8471fb6e58171882f3067)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:10.659740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:10.733433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:10.735222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:10.770367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd452efd6a8471fb6e58171882f3067'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.780785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfbc7f-a0ab-41a6-a797-5dc3d302d06c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:10.792185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd452efd6a8471fb6e58171882f3067'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.792477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:10.793291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd452efd6a8471fb6e58171882f3067'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.793395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.793542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.793822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] PATH_INFO: `/v3/projects/c4b484ab5466406a98e6e56ab4f8087f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:10.795475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4b484ab5466406a98e6e56ab4f8087f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:10.807194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:10.807833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd452efd6a8471fb6e58171882f3067'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.831258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef4914bf-39cf-4e05-ba93-9af78207e357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd452efd6a8471fb6e58171882f3067'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.841061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1050/2077] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:10 2022] DELETE /identity/v3/users/6dd452efd6a8471fb6e58171882f3067 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:10.846227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.846957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.847490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:10.873302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:10.874696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:10.882571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:10.891431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4b484ab5466406a98e6e56ab4f8087f', 'request_id': 'req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.902027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4b484ab5466406a98e6e56ab4f8087f', 'request_id': 'req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.909746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4b484ab5466406a98e6e56ab4f8087f', 'request_id': 'req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.926160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] The token cache is being invalidate because project c4b484ab5466406a98e6e56ab4f8087f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:10.926521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.927742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f5d01644-b95c-4ad6-86b8-ea7fae033dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:10.929412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1028/2078] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:10 2022] DELETE /identity/v3/projects/c4b484ab5466406a98e6e56ab4f8087f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:10.934673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0452f-0548-45a8-b3bc-c1d6924514bc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.935021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0452f-0548-45a8-b3bc-c1d6924514bc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.935265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0452f-0548-45a8-b3bc-c1d6924514bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:10.951394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1051/2079] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:32:10.955636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8fc0452f-0548-45a8-b3bc-c1d6924514bc None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:10.956957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8fc0452f-0548-45a8-b3bc-c1d6924514bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:10.959975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:10.961910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:10.966808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1029/2080] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:32:10.980968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:10.981254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:10.981527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.014257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:11.015825 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:11.024808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1030/2081] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:11.028883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.037543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf63be6-1b32-4d9b-9767-ba9bb050575c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.038404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee3388-a56a-4fda-94cd-f81c8310a25a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.044868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.045946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.046305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.046648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.046755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.047543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.047823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.048162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.048887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:11.049270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.050943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.058845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:11.061614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.063654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.070458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1031/2082] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:11.071457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.075416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1052/2083] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:32:11.100190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd5c69-c077-4733-a4dd-3b802b2660b1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.106626 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.108092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.108509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.108787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.109425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.111525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.143539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1032/2084] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:11 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:11.147677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.158444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bf287-f2a5-4b4b-a72f-04a42c0f7b85 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.248992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.250200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.250612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.251030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.251847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.256596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.300542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1053/2085] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:11 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:11.301760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.313267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6271a75-cbc2-40f1-bd49-49c873138a77 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.322530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.326319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.326319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.326319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.326319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.327799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.342228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1033/2086] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:11.343463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.355836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89582a2-3c00-49af-ab46-c949b05ebc53 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.363789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf9b7e-c92a-4047-b39f-e8b9f00dab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.364414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9b7e-c92a-4047-b39f-e8b9f00dab76 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.364698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9b7e-c92a-4047-b39f-e8b9f00dab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.364972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf9b7e-c92a-4047-b39f-e8b9f00dab76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.365624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf9b7e-c92a-4047-b39f-e8b9f00dab76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.367319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf9b7e-c92a-4047-b39f-e8b9f00dab76 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.383181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1054/2087] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:11.384407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.396407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af138b87-6eb4-47cc-973e-7d52e920ae70 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.402228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.403116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.403486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.403921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] PATH_INFO: `/v3/projects/e553d3c8c12647c9bced297af2db2dec/users/a933dbd0775b401b93b7afa9b5aa1adf/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.420981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e553d3c8c12647c9bced297af2db2dec, user_id=a933dbd0775b401b93b7afa9b5aa1adf, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.424367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.443025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1034/2088] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:11 2022] PUT /identity/v3/projects/e553d3c8c12647c9bced297af2db2dec/users/a933dbd0775b401b93b7afa9b5aa1adf/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:32:11.455274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.455637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.455928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.470703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.491627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:11.509781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] MFA Rules not processed for user `a933dbd0775b401b93b7afa9b5aa1adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:11.513104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:11.556737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:11.601758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8eb5ab-d93c-44dd-8945-1a22cb29367d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.610560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.611508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.611508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.611910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] PATH_INFO: `/v3/projects/ffb779bfa1034442b023362d56bab4ff` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.613135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb779bfa1034442b023362d56bab4ff)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.615283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.620530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1055/2089] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:11.632045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.634656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:11.683557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb779bfa1034442b023362d56bab4ff', 'request_id': 'req-1a734a67-195f-4293-9a85-07f770698318'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.691211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb779bfa1034442b023362d56bab4ff', 'request_id': 'req-1a734a67-195f-4293-9a85-07f770698318'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.701324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb779bfa1034442b023362d56bab4ff', 'request_id': 'req-1a734a67-195f-4293-9a85-07f770698318'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.708254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c09a2-b9a1-41bc-9a0a-fc94178acb83 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.717500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] The token cache is being invalidate because project ffb779bfa1034442b023362d56bab4ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:11.717784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.718503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.725474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1035/2090] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:11 2022] DELETE /identity/v3/projects/ffb779bfa1034442b023362d56bab4ff => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:11.726657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:11.729404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:11.770967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:11.788835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.789371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.789811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.790109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.791962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:11.819294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a734a67-195f-4293-9a85-07f770698318 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:11.825673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:11.826336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:11.826613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:11.827012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] PATH_INFO: `/v3/users/7f7dbf195f7e428192ae7d4c5c03287f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:11.828134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7dbf195f7e428192ae7d4c5c03287f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.830076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.880033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:11.882128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:11.894329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1056/2091] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:32:11.902629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7dbf195f7e428192ae7d4c5c03287f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.910748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7dbf195f7e428192ae7d4c5c03287f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.927839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7dbf195f7e428192ae7d4c5c03287f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.928968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7dbf195f7e428192ae7d4c5c03287f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.950205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b3943dec-03bd-489f-8e4b-fe67f3917c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7dbf195f7e428192ae7d4c5c03287f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:11.959427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1036/2092] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:11 2022] DELETE /identity/v3/users/7f7dbf195f7e428192ae7d4c5c03287f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:12.373305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:12.390246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdcab2-aa5a-4256-8a16-d8bec6da5081 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:12.482737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:12.542632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:12.543778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:12.544715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:12.546233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] PATH_INFO: `/v3/projects/3ea5d45056244861b65a2dc1e8972858` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:12.547587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ea5d45056244861b65a2dc1e8972858)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:12.551006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:12.638733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ea5d45056244861b65a2dc1e8972858', 'request_id': 'req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:12.645218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ea5d45056244861b65a2dc1e8972858', 'request_id': 'req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:12.652516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ea5d45056244861b65a2dc1e8972858', 'request_id': 'req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:12.672432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] The token cache is being invalidate because project 3ea5d45056244861b65a2dc1e8972858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:12.672769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:12.673523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-1bcc42f1-80de-4ca2-baae-54dd5eec44ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:12.675338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1057/2093] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:12 2022] DELETE /identity/v3/projects/3ea5d45056244861b65a2dc1e8972858 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:12.681449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:12.683703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:12.684200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:12.714895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:12.716506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:12.780083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1037/2094] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:32:12.788196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:12.788557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:12.788842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:12.812514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:12.814164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:12.825434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1058/2095] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:12.830354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:12.841496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21c5f7-a38d-413a-90ab-82e6facf371e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:12.846617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:12.847340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:12.847853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:12.848246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:12.849138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:12.849355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:12.851273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:12.875979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1038/2096] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:12.876399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:12.886650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3ff90-6e02-4410-85e6-3a1c4e05d137 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:12.893751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:12.894357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:12.894655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:12.894921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:12.895654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:12.897642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:12.931723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1059/2097] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:12 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:32:12.936143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:12.957856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170af6a-28a8-4ad2-9cfe-0a78cb421be3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:12.997453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:13.021869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.022472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.022751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.023023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.023774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.025726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.052229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1039/2098] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:12 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:32:13.056433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.067019 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf3b81-7423-4474-b7b9-a03cd090bbf2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.072099 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.072727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.073011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.073283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.073929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.075604 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.085789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1060/2099] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:13.089809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.099926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d270da-193b-4b98-82b1-d453c4806375 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.105178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0959476-7c3d-4da0-aadd-168b6388deea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.106028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959476-7c3d-4da0-aadd-168b6388deea admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.106313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959476-7c3d-4da0-aadd-168b6388deea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.106735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959476-7c3d-4da0-aadd-168b6388deea admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.107672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0959476-7c3d-4da0-aadd-168b6388deea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.109559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0959476-7c3d-4da0-aadd-168b6388deea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.122401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1040/2100] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:13.126534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.136741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9303d12-ef3a-47d4-b362-22aea73b95ee admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.141397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.142054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.142336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.142612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] PATH_INFO: `/v3/projects/9f77f52133cb4672b07d95ca65ed2b17/users/6105c16eac7349f7a118cc0f8e20801b/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.156864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f77f52133cb4672b07d95ca65ed2b17, user_id=6105c16eac7349f7a118cc0f8e20801b, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.159490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.173006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1061/2101] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:13 2022] PUT /identity/v3/projects/9f77f52133cb4672b07d95ca65ed2b17/users/6105c16eac7349f7a118cc0f8e20801b/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:13.184733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e41367-799f-4151-b436-0a0e3874dada None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.185029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e41367-799f-4151-b436-0a0e3874dada None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.185309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e41367-799f-4151-b436-0a0e3874dada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.213263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-22e41367-799f-4151-b436-0a0e3874dada None None] MFA Rules not processed for user `6105c16eac7349f7a118cc0f8e20801b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:13.214593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-22e41367-799f-4151-b436-0a0e3874dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:13.281651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1041/2102] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:13.286106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.350181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8724a22-3196-40db-a17a-ead8aa23986b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.355819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.356513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.356839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.357150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.357820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.359791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.364265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:13.388358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1062/2103] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:13 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:13.392844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e41367-799f-4151-b436-0a0e3874dada None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.404195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e41367-799f-4151-b436-0a0e3874dada None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.463769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.464280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.464559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.464829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.465452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.467380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.493265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1042/2104] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:13 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:13.494257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.505709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85446b4-f995-4cd4-b4c2-be5c1474c20e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.510947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.511610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.511887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.512161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.512842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.514495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.530586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1063/2105] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:13.535832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.551467 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595dec8-19b4-475a-b04f-021322790ffa admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.557410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.558121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.558423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.558699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] PATH_INFO: `/v3/projects/42f8f83376714170878c8aadf6645742/users/54489c6ccf12471584f89bc8bd5d8bd8/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.576579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f8f83376714170878c8aadf6645742, user_id=54489c6ccf12471584f89bc8bd5d8bd8, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.579359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.595322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1043/2106] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:13 2022] PUT /identity/v3/projects/42f8f83376714170878c8aadf6645742/users/54489c6ccf12471584f89bc8bd5d8bd8/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:32:13.599461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.611339 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639418b-7ebd-4d5c-901e-0cbb6094a07a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.674344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2952ba-3c94-4b65-8551-af96693e1b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.675109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2952ba-3c94-4b65-8551-af96693e1b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.675263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2952ba-3c94-4b65-8551-af96693e1b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.675538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2952ba-3c94-4b65-8551-af96693e1b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.676194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2952ba-3c94-4b65-8551-af96693e1b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.677799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2952ba-3c94-4b65-8551-af96693e1b9e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.693250 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1064/2107] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:13.699211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.712733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f73a7a4-d5b3-4792-90d5-3bba9e40c535 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.717662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.718327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.718608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.718879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] PATH_INFO: `/v3/projects/42f8f83376714170878c8aadf6645742/users/54489c6ccf12471584f89bc8bd5d8bd8/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.721498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f8f83376714170878c8aadf6645742, user_id=54489c6ccf12471584f89bc8bd5d8bd8, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.724885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:13.737450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:13.743535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1044/2108] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:13 2022] PUT /identity/v3/projects/42f8f83376714170878c8aadf6645742/users/54489c6ccf12471584f89bc8bd5d8bd8/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:13.755680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aca375-f38b-4470-b533-05793c20db53 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.755966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aca375-f38b-4470-b533-05793c20db53 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.756248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aca375-f38b-4470-b533-05793c20db53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.789439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-42aca375-f38b-4470-b533-05793c20db53 None None] MFA Rules not processed for user `54489c6ccf12471584f89bc8bd5d8bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:13.791820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-42aca375-f38b-4470-b533-05793c20db53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:13.880799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1065/2109] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:13.892961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:13.894816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:13.967391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d47ac8-281f-4fdc-98bd-22161e3f7556 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:13.972028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718e16c-6d0c-4e0c-8e3d-3d468245428d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:13.972587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e16c-6d0c-4e0c-8e3d-3d468245428d service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:13.972873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e16c-6d0c-4e0c-8e3d-3d468245428d service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:13.973763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718e16c-6d0c-4e0c-8e3d-3d468245428d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:13.987681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718e16c-6d0c-4e0c-8e3d-3d468245428d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:13.989778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718e16c-6d0c-4e0c-8e3d-3d468245428d service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:14.004260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1045/2110] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:14.059761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aca375-f38b-4470-b533-05793c20db53 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:14.181597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:14.188632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aca375-f38b-4470-b533-05793c20db53 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:14.202679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:14.203419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:14.203699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:14.203969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] PATH_INFO: `/v3/users/6105c16eac7349f7a118cc0f8e20801b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:14.205186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6105c16eac7349f7a118cc0f8e20801b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:14.207386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:14.255932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6105c16eac7349f7a118cc0f8e20801b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.264469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6105c16eac7349f7a118cc0f8e20801b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.265189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6105c16eac7349f7a118cc0f8e20801b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.272308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6105c16eac7349f7a118cc0f8e20801b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.290104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6105c16eac7349f7a118cc0f8e20801b'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.297848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1066/2111] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:14 2022] DELETE /identity/v3/users/6105c16eac7349f7a118cc0f8e20801b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:14.302474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:14.302803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:14.303079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:14.324208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:14.325665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:14.401932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1046/2112] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:14.413616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:14.416570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:14.496510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a892655-4951-4963-b727-9ac51fdb050d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:14.500413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:14.501022 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:14.501304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:14.501580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:14.512004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:14.513408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:14.526443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1067/2113] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:14.875496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:14.878139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:14.891613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacb31e-5ad6-466f-9ed8-51550709a495 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:14.896828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:14.897505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:14.898029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:14.898806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] PATH_INFO: `/v3/projects/9f77f52133cb4672b07d95ca65ed2b17` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:14.900050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f77f52133cb4672b07d95ca65ed2b17)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:14.902197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:14.960651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f77f52133cb4672b07d95ca65ed2b17', 'request_id': 'req-ddd5817a-09d6-4f15-bc88-57c6601a2d10'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.966956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f77f52133cb4672b07d95ca65ed2b17', 'request_id': 'req-ddd5817a-09d6-4f15-bc88-57c6601a2d10'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.973390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f77f52133cb4672b07d95ca65ed2b17', 'request_id': 'req-ddd5817a-09d6-4f15-bc88-57c6601a2d10'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.987108 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] The token cache is being invalidate because project 9f77f52133cb4672b07d95ca65ed2b17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:14.987550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.988319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:14.989963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1047/2114] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:14 2022] DELETE /identity/v3/projects/9f77f52133cb4672b07d95ca65ed2b17 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:14.994492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:15.004523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f0f91-d3cd-4fcb-bff5-4dad372e6767 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:15.066773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:15.067425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:15.067707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:15.067983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] PATH_INFO: `/v3/users/54489c6ccf12471584f89bc8bd5d8bd8` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:15.069087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54489c6ccf12471584f89bc8bd5d8bd8)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:15.071580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:15.092591 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:15.120563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54489c6ccf12471584f89bc8bd5d8bd8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.129147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54489c6ccf12471584f89bc8bd5d8bd8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.130018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54489c6ccf12471584f89bc8bd5d8bd8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.146831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54489c6ccf12471584f89bc8bd5d8bd8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.160592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6a9d1d83-ec69-4558-892d-083519cd8ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54489c6ccf12471584f89bc8bd5d8bd8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.168647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1068/2115] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:14 2022] DELETE /identity/v3/users/54489c6ccf12471584f89bc8bd5d8bd8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:15.541133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:15.543154 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:15.562877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5817a-09d6-4f15-bc88-57c6601a2d10 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:15.589405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:15.649517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:15.650618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:15.650819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:15.651074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] PATH_INFO: `/v3/projects/42f8f83376714170878c8aadf6645742` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:15.652726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f8f83376714170878c8aadf6645742)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:15.655043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:15.731630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f8f83376714170878c8aadf6645742', 'request_id': 'req-3d8cf82d-b2d3-47c1-945d-8690826e1a87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.737801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f8f83376714170878c8aadf6645742', 'request_id': 'req-3d8cf82d-b2d3-47c1-945d-8690826e1a87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.745046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f8f83376714170878c8aadf6645742', 'request_id': 'req-3d8cf82d-b2d3-47c1-945d-8690826e1a87'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.758163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] The token cache is being invalidate because project 42f8f83376714170878c8aadf6645742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:15.758452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.759062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3d8cf82d-b2d3-47c1-945d-8690826e1a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:15.760570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1048/2116] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:15 2022] DELETE /identity/v3/projects/42f8f83376714170878c8aadf6645742 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:15.766772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:15.767056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:15.767363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:15.786780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:15.787927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:15.846716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1069/2117] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:15.855207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:15.855489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:15.855764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:15.883627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:15.884862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:15.893545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1049/2118] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:15.897868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:15.908302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a01796-b2d9-4b64-9c6f-b66cf2098221 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:15.913337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:15.914426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:15.914710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:15.914981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:15.915688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:15.916288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:15.917901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:15.930585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1070/2119] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:15.934923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:15.944471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c718677-92f8-440b-b4f2-4e3c9392905e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:15.949699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:15.950498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:15.950780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:15.951076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:15.951713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:15.953641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:15.988512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1050/2120] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:15 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:15.989610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:16.000901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ceb13-775e-4645-94a9-957d6ed1a7df admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:16.068834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:16.071209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:16.071209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:16.071433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:16.072518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:16.075737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:16.107850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1071/2121] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:15 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:32:16.111752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:16.125661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97066186-a8a7-477e-84f6-34d75f768cab admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:16.131049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:16.131380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:16.131594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:16.131873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:16.132555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:16.134853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:16.145667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1051/2122] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:32:16.149859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:16.160125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1ee6-1599-4fbe-9bfb-15b8d6c9e3cc admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:16.165347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3100089-c30d-40cb-a885-f972e0cbb685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:16.165943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3100089-c30d-40cb-a885-f972e0cbb685 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:16.166217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3100089-c30d-40cb-a885-f972e0cbb685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:16.166548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3100089-c30d-40cb-a885-f972e0cbb685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:16.167160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3100089-c30d-40cb-a885-f972e0cbb685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:16.168739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3100089-c30d-40cb-a885-f972e0cbb685 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:16.184977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1072/2123] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:16.186042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:16.197087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd9777-1f50-48af-a559-1451d6bd4436 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:16.201861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:16.202590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:16.202938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:16.203333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] PATH_INFO: `/v3/projects/e6551ae647e5470e971bfa089afc10e4/users/2414021316e24ff887ce3472689b7d51/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:16.219177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6551ae647e5470e971bfa089afc10e4, user_id=2414021316e24ff887ce3472689b7d51, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:16.221852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:16.235916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1052/2124] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:16 2022] PUT /identity/v3/projects/e6551ae647e5470e971bfa089afc10e4/users/2414021316e24ff887ce3472689b7d51/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:32:16.247588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678ecea-19d6-4987-99af-0b61993522d7 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:16.247892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678ecea-19d6-4987-99af-0b61993522d7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:16.248211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678ecea-19d6-4987-99af-0b61993522d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:16.256105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:16.279650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b678ecea-19d6-4987-99af-0b61993522d7 None None] MFA Rules not processed for user `2414021316e24ff887ce3472689b7d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:16.280856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b678ecea-19d6-4987-99af-0b61993522d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:16.344804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1073/2125] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:16.356233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:16.358095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:16.416072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eeefb7-d58c-42c1-a9a9-4a6ddf5a8a3d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:16.420025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30312a53-2b56-4809-9cee-279d1ba4f229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:16.420623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30312a53-2b56-4809-9cee-279d1ba4f229 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:16.420898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30312a53-2b56-4809-9cee-279d1ba4f229 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:16.421166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30312a53-2b56-4809-9cee-279d1ba4f229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:16.426303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:16.433326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30312a53-2b56-4809-9cee-279d1ba4f229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:16.435053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30312a53-2b56-4809-9cee-279d1ba4f229 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:16.445777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1053/2126] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:17.840421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c581-6777-4086-a5f2-9892e6c4f5a1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:17.841121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c581-6777-4086-a5f2-9892e6c4f5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:17.841121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c581-6777-4086-a5f2-9892e6c4f5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:17.877205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bce7c581-6777-4086-a5f2-9892e6c4f5a1 None None] MFA Rules not processed for user `2414021316e24ff887ce3472689b7d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:17.878996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bce7c581-6777-4086-a5f2-9892e6c4f5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:17.901289 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1074/2127] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:21.309429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ccdf-5687-4f4c-a69d-76e923f468de None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:21.310348 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ccdf-5687-4f4c-a69d-76e923f468de None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:21.310585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ccdf-5687-4f4c-a69d-76e923f468de None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:21.312415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1054/2128] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:32:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:32:21.339233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:21.339851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:21.339851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:21.360972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:21.362333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:21.376728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ccdf-5687-4f4c-a69d-76e923f468de None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:21.378512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-eab6ccdf-5687-4f4c-a69d-76e923f468de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:21.434270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1075/2129] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:21.445489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:21.447450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:21.449004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ccdf-5687-4f4c-a69d-76e923f468de None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:21.453503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:21.454193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:21.454548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:21.454889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] PATH_INFO: `/v3/users/33c71ba6b2ad438d8593fa19eb617a34` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:21.456057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c71ba6b2ad438d8593fa19eb617a34)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:21.457982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:21.459962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51954d5c-16da-4c2e-a59e-69716737c4c5 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:21.466372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:21.466979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:21.467286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:21.467560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:21.476296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:21.477986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:21.489753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1076/2130] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:32:21.512449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c71ba6b2ad438d8593fa19eb617a34'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.521014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c71ba6b2ad438d8593fa19eb617a34'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.530844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c71ba6b2ad438d8593fa19eb617a34'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.531521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c71ba6b2ad438d8593fa19eb617a34'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.547637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-b4ab3f1a-21fe-4af7-970c-57ebb539bd41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c71ba6b2ad438d8593fa19eb617a34'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.553522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1055/2131] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:21 2022] DELETE /identity/v3/users/33c71ba6b2ad438d8593fa19eb617a34 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:21.738729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:21.750432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dad5b-02b8-4355-9f70-8ce26c3c5c30 service placement] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:21.808102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:21.808728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:21.809008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:21.809282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] PATH_INFO: `/v3/projects/82cfe2f487fc4393bda828dc28471dca` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:21.810522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82cfe2f487fc4393bda828dc28471dca)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:21.812489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:21.823483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:21.859027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82cfe2f487fc4393bda828dc28471dca', 'request_id': 'req-3dc755ee-7fc0-4f34-84bf-b64b741ac529'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.864439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82cfe2f487fc4393bda828dc28471dca', 'request_id': 'req-3dc755ee-7fc0-4f34-84bf-b64b741ac529'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.871150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82cfe2f487fc4393bda828dc28471dca', 'request_id': 'req-3dc755ee-7fc0-4f34-84bf-b64b741ac529'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.884933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] The token cache is being invalidate because project 82cfe2f487fc4393bda828dc28471dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:21.885225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.885839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3dc755ee-7fc0-4f34-84bf-b64b741ac529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:21.887344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1077/2132] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:21 2022] DELETE /identity/v3/projects/82cfe2f487fc4393bda828dc28471dca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:21.892496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:21.892785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:21.893058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:21.912874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:21.919048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:21.997866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1056/2133] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:22.005910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.006205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.006488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.027498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:22.028706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:22.036777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1078/2134] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:22.041507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.051461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed65cd9-292a-4a4b-b17d-ac12f6f64395 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.056276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.056875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.057157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.057457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.058105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:22.058413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.060097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.068828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:22.076909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1057/2135] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:22.078221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.088490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0440fc6-e33d-455c-aa52-c48679e23948 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.093334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.094299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.094782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.095312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.096451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.100248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.142463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1079/2136] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:22 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:22.143731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.153892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad4795-4854-4fa5-bdd2-9a8d7ebb4645 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.210263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.210867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.211183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.211952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.212580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.214513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.238116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1058/2137] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:22 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:22.243261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.253547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e1d19-c049-4297-982e-46802c63a3e6 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.258263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.258864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.259181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.259459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.260102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.261707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.271620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1080/2138] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:22.276068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.286267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f308bc-705f-4dc6-a275-336ea213497b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.291908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a8a57-b911-40af-bac3-eab53b08b608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.292512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a8a57-b911-40af-bac3-eab53b08b608 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.292796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a8a57-b911-40af-bac3-eab53b08b608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.293074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a8a57-b911-40af-bac3-eab53b08b608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.294006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a8a57-b911-40af-bac3-eab53b08b608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.295661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a8a57-b911-40af-bac3-eab53b08b608 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.307895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1059/2139] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:22.312498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.323313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a926d6-6582-4e8a-8de7-09f874c1ccbd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.328492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.329183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.329494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.329823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] PATH_INFO: `/v3/projects/6ecb83fbcae94dca82331ab87f028878/users/7d373dfa8c474ade92e94dc73dd9926f/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.345609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecb83fbcae94dca82331ab87f028878, user_id=7d373dfa8c474ade92e94dc73dd9926f, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.348296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.362353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1081/2140] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:22 2022] PUT /identity/v3/projects/6ecb83fbcae94dca82331ab87f028878/users/7d373dfa8c474ade92e94dc73dd9926f/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:32:22.367633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09041b-7329-42e6-9227-4260176b6aa8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.368079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09041b-7329-42e6-9227-4260176b6aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.368266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09041b-7329-42e6-9227-4260176b6aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.387114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5f09041b-7329-42e6-9227-4260176b6aa8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:22.388346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5f09041b-7329-42e6-9227-4260176b6aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:22.447719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1060/2141] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:22.457416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:22.459178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:22.516758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffece1-c8c7-459d-82fe-f6c293f20a23 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:22.520341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ad8e6-4a1c-48bb-afad-3b39ccf8cc05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:22.520939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad8e6-4a1c-48bb-afad-3b39ccf8cc05 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:22.521228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad8e6-4a1c-48bb-afad-3b39ccf8cc05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:22.521496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ad8e6-4a1c-48bb-afad-3b39ccf8cc05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:22.531424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ad8e6-4a1c-48bb-afad-3b39ccf8cc05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:22.532867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ad8e6-4a1c-48bb-afad-3b39ccf8cc05 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:22.545366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1082/2142] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:27.794954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:27.795536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:27.796086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:27.835552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] MFA Rules not processed for user `7d373dfa8c474ade92e94dc73dd9926f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:27.837449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:27.912682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1061/2143] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:27.924732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdd7b5-cb79-4cce-ab0b-a24dc50164d4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:27.925280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdd7b5-cb79-4cce-ab0b-a24dc50164d4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:27.925280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefdd7b5-cb79-4cce-ab0b-a24dc50164d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:27.945767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-eefdd7b5-cb79-4cce-ab0b-a24dc50164d4 None None] MFA Rules not processed for user `7d373dfa8c474ade92e94dc73dd9926f`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:27.947189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-eefdd7b5-cb79-4cce-ab0b-a24dc50164d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:27.954441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1083/2144] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:27.965616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:27.967032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:27.980262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:28.047471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4bb5c-8fc7-461b-91a1-39d47651f3ca None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:28.060188 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a37a3d-82fe-4b46-99d8-b5fb58fe0066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:28.060819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a37a3d-82fe-4b46-99d8-b5fb58fe0066 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:28.061101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a37a3d-82fe-4b46-99d8-b5fb58fe0066 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:28.061376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a37a3d-82fe-4b46-99d8-b5fb58fe0066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:28.074291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a37a3d-82fe-4b46-99d8-b5fb58fe0066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:28.074291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a37a3d-82fe-4b46-99d8-b5fb58fe0066 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:28.089654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1062/2145] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:28.158532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e67e-bc4d-40d2-9019-debb8d02704d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:28.158956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e67e-bc4d-40d2-9019-debb8d02704d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:28.159473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754e67e-bc4d-40d2-9019-debb8d02704d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:28.188770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9754e67e-bc4d-40d2-9019-debb8d02704d None None] MFA Rules not processed for user `c29b21ee2cd341169a2c8263393c5db6`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:28.190093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9754e67e-bc4d-40d2-9019-debb8d02704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:28.252468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:28.321566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1084/2146] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:29.743625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c0b8f-5058-421c-913a-5ce5df859609 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:29.744102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c0b8f-5058-421c-913a-5ce5df859609 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:29.744304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c0b8f-5058-421c-913a-5ce5df859609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:29.786774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cc6c0b8f-5058-421c-913a-5ce5df859609 None None] MFA Rules not processed for user `7d373dfa8c474ade92e94dc73dd9926f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:29.788961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cc6c0b8f-5058-421c-913a-5ce5df859609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:29.802866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1063/2147] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:29.874683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:29.875207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:29.875290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:29.902518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] MFA Rules not processed for user `2495d3c63dc44049872afb908ca6133b`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:29.903851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:29.989956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1085/2148] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:39.778584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=33 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:39.973685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=36 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:41.960992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e41e74-302a-4eb8-b7c4-68187b0daddd None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:41.960992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e41e74-302a-4eb8-b7c4-68187b0daddd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:41.960992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e41e74-302a-4eb8-b7c4-68187b0daddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:41.977871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-26e41e74-302a-4eb8-b7c4-68187b0daddd None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:41.978987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-26e41e74-302a-4eb8-b7c4-68187b0daddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:42.033587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1064/2149] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:32:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:42.077533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:42.078997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:42.153892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86142628-6692-43e7-9739-bd69f80a1d13 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:42.158963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:42.159601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:42.159883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:42.160159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:42.169157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:42.170771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:42.180157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1086/2150] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:32:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:32:42.759743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a88b3-bcaa-4199-90f4-283e2087435b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:42.760032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a88b3-bcaa-4199-90f4-283e2087435b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:42.760359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a88b3-bcaa-4199-90f4-283e2087435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:42.780469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9b3a88b3-bcaa-4199-90f4-283e2087435b None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:42.781683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9b3a88b3-bcaa-4199-90f4-283e2087435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:42.798736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1065/2151] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 14:32:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:42.814765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:42.827350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841817d-63ca-4ea6-9169-ddea6d3ea716 service swift] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:42.834320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:42.835011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:42.835393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:42.836450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:42.847063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:42.848805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:42.862392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1087/2152] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:32:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:32:43.137112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6979d9-ddbe-43aa-acdf-095ef0942564 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:43.137560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6979d9-ddbe-43aa-acdf-095ef0942564 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:43.137645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6979d9-ddbe-43aa-acdf-095ef0942564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:43.157054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2b6979d9-ddbe-43aa-acdf-095ef0942564 None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:43.158497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2b6979d9-ddbe-43aa-acdf-095ef0942564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:43.167589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1066/2153] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:43.180082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:43.191771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b83da-3fb3-41b3-8998-999059dffb63 service swift] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:43.197518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:43.198109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:43.198388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:43.198663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:43.206747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:43.208358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:43.216207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1088/2154] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:32:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:32:43.540225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:43.540508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:43.540802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:43.559816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:43.561007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:43.568179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1067/2155] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 14:32:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:43.581290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:43.593643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a062899-f90b-4582-bc02-e7c3f56a8d82 service swift] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:43.599037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5eea9-5a5f-4c86-9ee3-4911e62222a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:43.599841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5eea9-5a5f-4c86-9ee3-4911e62222a6 service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:43.600244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5eea9-5a5f-4c86-9ee3-4911e62222a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:43.600523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5eea9-5a5f-4c86-9ee3-4911e62222a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:43.609593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5eea9-5a5f-4c86-9ee3-4911e62222a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:43.611281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5eea9-5a5f-4c86-9ee3-4911e62222a6 service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:43.620253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1089/2156] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:32:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:32:45.901574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:45.903177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:45.923084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6733ac9-288e-4b10-bef7-11a69974be6c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:45.935453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:45.936391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:45.936885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:45.937342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] PATH_INFO: `/v3/users/2414021316e24ff887ce3472689b7d51` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:45.938830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2414021316e24ff887ce3472689b7d51)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:45.941034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:46.012207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2414021316e24ff887ce3472689b7d51'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.021184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2414021316e24ff887ce3472689b7d51'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.033023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2414021316e24ff887ce3472689b7d51'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.034080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2414021316e24ff887ce3472689b7d51'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.051791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2414021316e24ff887ce3472689b7d51'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.081541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1068/2157] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:45 2022] DELETE /identity/v3/users/2414021316e24ff887ce3472689b7d51 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:32:46.083356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.084517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.084737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.122766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:46.124897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:46.235875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1090/2158] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:46.256846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:46.258920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:46.354162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7922f44b-40f6-405e-a1fa-de41aa560d08 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:46.359888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e929a3-8be2-42b5-845b-fd53e05e25de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:46.360526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e929a3-8be2-42b5-845b-fd53e05e25de service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.360813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e929a3-8be2-42b5-845b-fd53e05e25de service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.361092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e929a3-8be2-42b5-845b-fd53e05e25de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.372845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e929a3-8be2-42b5-845b-fd53e05e25de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:46.374319 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e929a3-8be2-42b5-845b-fd53e05e25de service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:46.380461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:46.392369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1069/2159] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:46.607727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:46.609248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:46.629086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92d4c0-993d-4032-b9b0-b14e872ae955 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:46.635733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:46.636659 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.637205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.637732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] PATH_INFO: `/v3/projects/e6551ae647e5470e971bfa089afc10e4` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.639154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6551ae647e5470e971bfa089afc10e4)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:46.641586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:46.714500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:46.718241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6551ae647e5470e971bfa089afc10e4', 'request_id': 'req-69ac7bc9-21e2-4710-8090-99416e132ef9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.725865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6551ae647e5470e971bfa089afc10e4', 'request_id': 'req-69ac7bc9-21e2-4710-8090-99416e132ef9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.733047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6551ae647e5470e971bfa089afc10e4', 'request_id': 'req-69ac7bc9-21e2-4710-8090-99416e132ef9'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.746054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] The token cache is being invalidate because project e6551ae647e5470e971bfa089afc10e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:46.746350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.746979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-69ac7bc9-21e2-4710-8090-99416e132ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:46.748495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1091/2160] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:46 2022] DELETE /identity/v3/projects/e6551ae647e5470e971bfa089afc10e4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:46.753377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.753666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.753990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.779181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:46.780405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:46.839409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1070/2161] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:46.847152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.847469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.847741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.867901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:46.869121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:46.877263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1092/2162] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:46.881666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:46.891795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66517448-43ce-4f8a-93c0-a615c23d68c6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:46.896270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:46.896860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.897176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.897446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.898738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:46.899040 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:46.900649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:46.912661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1071/2163] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:46.917152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:46.927170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db346e6-8672-473a-88af-280fb8e3a2da None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:46.932312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:46.932901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:46.933177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:46.933443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:46.934065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:46.935996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:46.963316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1093/2164] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:46 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:46.967695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:46.978016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfbbc2f-0b9c-4f2a-bfda-34cc615e726f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:47.042105 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:47.042828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:47.043551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:47.043905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:47.044602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:47.046601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:47.071859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1072/2165] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:46 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:47.075611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:47.087940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07796f8a-e956-4c2a-ba8e-9ce88e467dcd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:47.094199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:47.094817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:47.095113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:47.095390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:47.096036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:47.097715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:47.110733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1094/2166] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:47.114775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:47.125965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33296ed5-c608-4c8d-a369-dc88f35b77b6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:47.131135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe67d90-6191-4241-a6c6-545aa526f5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:47.131821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe67d90-6191-4241-a6c6-545aa526f5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:47.132103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe67d90-6191-4241-a6c6-545aa526f5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:47.132436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe67d90-6191-4241-a6c6-545aa526f5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:47.133229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe67d90-6191-4241-a6c6-545aa526f5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:47.135082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe67d90-6191-4241-a6c6-545aa526f5bf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:47.146930 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1073/2167] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:32:47.151929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:47.163135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41bed1-25f7-42a0-9c54-f5deb40900c5 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:47.169701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:47.170402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:47.170665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:47.170943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] PATH_INFO: `/v3/projects/e9bff7b0f0f542baa6c6ce5855cf212d/users/2d347f428d6e495eae08d35c350b6054/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:47.191975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bff7b0f0f542baa6c6ce5855cf212d, user_id=2d347f428d6e495eae08d35c350b6054, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:47.196032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:47.215447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1095/2168] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:47 2022] PUT /identity/v3/projects/e9bff7b0f0f542baa6c6ce5855cf212d/users/2d347f428d6e495eae08d35c350b6054/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:47.226004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:47.226364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:47.226689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:47.266715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] MFA Rules not processed for user `2d347f428d6e495eae08d35c350b6054`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:47.268700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:47.340673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:47.351635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1074/2169] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:47.364519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:47.371399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:47.445411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb87ec-d3a5-42e3-bbeb-18e8fc9c16c7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:47.450276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad29e04-a8eb-432f-925a-fec6cc7744cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:47.450898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad29e04-a8eb-432f-925a-fec6cc7744cb service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:47.451985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad29e04-a8eb-432f-925a-fec6cc7744cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:47.452072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad29e04-a8eb-432f-925a-fec6cc7744cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:47.480930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad29e04-a8eb-432f-925a-fec6cc7744cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:47.482392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad29e04-a8eb-432f-925a-fec6cc7744cb service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:47.493161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1096/2170] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:53.404404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:53.405854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:53.472489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3f9e6-400c-4f5b-a776-c5b094c2fa21 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:53.477315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:53.477869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:53.478143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:53.478409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] PATH_INFO: `/v3/users/a933dbd0775b401b93b7afa9b5aa1adf` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:53.479548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a933dbd0775b401b93b7afa9b5aa1adf)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:53.481532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:53.529571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a933dbd0775b401b93b7afa9b5aa1adf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:53.536852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a933dbd0775b401b93b7afa9b5aa1adf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:53.544864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a933dbd0775b401b93b7afa9b5aa1adf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:53.545491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a933dbd0775b401b93b7afa9b5aa1adf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:53.557585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a933dbd0775b401b93b7afa9b5aa1adf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:53.563716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1075/2171] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:32:53 2022] DELETE /identity/v3/users/a933dbd0775b401b93b7afa9b5aa1adf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:53.568104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:53.568617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:53.569028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:53.589063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:53.590573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:53.652361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1097/2172] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:53.663262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:53.664675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:53.721941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f42b23-ac62-4c3f-b80f-b7f9caa7235a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:53.725612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7cb039-f8b7-4d99-9316-387ff890a2ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:53.726239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb039-f8b7-4d99-9316-387ff890a2ce service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:53.726520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb039-f8b7-4d99-9316-387ff890a2ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:53.726794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cb039-f8b7-4d99-9316-387ff890a2ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:53.736324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cb039-f8b7-4d99-9316-387ff890a2ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:53.737713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cb039-f8b7-4d99-9316-387ff890a2ce service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:53.747690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1076/2173] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:54.023050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.024976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:54.037650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60877191-e2d9-47a1-946b-ee75d1872005 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.042720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.043640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.043920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.044215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] PATH_INFO: `/v3/projects/e553d3c8c12647c9bced297af2db2dec` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.045436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e553d3c8c12647c9bced297af2db2dec)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.047536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.056950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:54.099112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e553d3c8c12647c9bced297af2db2dec', 'request_id': 'req-55455d33-eab0-4056-91e1-1b2cbc76206e'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:54.105344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e553d3c8c12647c9bced297af2db2dec', 'request_id': 'req-55455d33-eab0-4056-91e1-1b2cbc76206e'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:54.111682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e553d3c8c12647c9bced297af2db2dec', 'request_id': 'req-55455d33-eab0-4056-91e1-1b2cbc76206e'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:54.124774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] The token cache is being invalidate because project e553d3c8c12647c9bced297af2db2dec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:32:54.125064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:54.125703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-55455d33-eab0-4056-91e1-1b2cbc76206e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:32:54.127246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1098/2174] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:32:54 2022] DELETE /identity/v3/projects/e553d3c8c12647c9bced297af2db2dec => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:54.132190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.132474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.132778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.151894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:54.153086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:54.165457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:54.215561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1077/2175] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:32:54.223355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.223662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.223941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.245461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:54.246671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:54.254618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1099/2176] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:54.258765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.268557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c252895-c3ed-41bc-82da-fe4adece0f66 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.273058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.273651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.273928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.274201 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.274840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:32:54.275207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.277073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.289007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1078/2177] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:32:54.293432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.302769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62e49f-8438-4e77-a89c-01166dc9609b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.308324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.308918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.309194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.309460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.310086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.312178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.340111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1100/2178] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:32:54 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:54.344440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.354899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c33878-a522-45df-9ced-1eb143721c68 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.415646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.416274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.416525 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.416800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.417425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.419668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.443312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1079/2179] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:32:54 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:32:54.446799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.459079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa8cfbd-a551-49a6-aada-8feeb0dafdd3 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.463196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.463932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.464210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.464479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.465174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.466795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.478428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1101/2180] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:54.482543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.493200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aaede1-9210-4035-8651-1a7850c7e82e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.499193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88576f-4415-414e-b1ab-8700a0157d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.499703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88576f-4415-414e-b1ab-8700a0157d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.499985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88576f-4415-414e-b1ab-8700a0157d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.500257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88576f-4415-414e-b1ab-8700a0157d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.500911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88576f-4415-414e-b1ab-8700a0157d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.502509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88576f-4415-414e-b1ab-8700a0157d07 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.513468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1080/2181] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:32:54.517799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.528359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bda8a-5ea4-4339-a53d-4d24d185aad2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.532874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.533512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.533790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.534063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] PATH_INFO: `/v3/projects/693b1fece5b548c786a495d2dc91bd8a/users/57e1243ffa724189ba7d2b9277e10afd/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.549960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693b1fece5b548c786a495d2dc91bd8a, user_id=57e1243ffa724189ba7d2b9277e10afd, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.552634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.565533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1102/2182] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:32:54 2022] PUT /identity/v3/projects/693b1fece5b548c786a495d2dc91bd8a/users/57e1243ffa724189ba7d2b9277e10afd/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:32:54.570869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094820-6f82-4650-ab66-f3c9537ac7c8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.571191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094820-6f82-4650-ab66-f3c9537ac7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.571471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad094820-6f82-4650-ab66-f3c9537ac7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.590874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ad094820-6f82-4650-ab66-f3c9537ac7c8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:54.592115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ad094820-6f82-4650-ab66-f3c9537ac7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:54.651112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1081/2183] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:32:54.660744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:54.662475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:54.720781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de215d-b1c4-4a15-a898-f15b090e4049 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:54.724472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:54.725078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:54.725367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:54.725631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:54.736161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:54.737586 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:54.747072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1103/2184] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:55.610504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535cf99-7e17-4dd6-87d1-9eba21667e54 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:55.611008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535cf99-7e17-4dd6-87d1-9eba21667e54 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:55.611133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535cf99-7e17-4dd6-87d1-9eba21667e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:55.640701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b535cf99-7e17-4dd6-87d1-9eba21667e54 None None] MFA Rules not processed for user `57e1243ffa724189ba7d2b9277e10afd`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:55.641945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b535cf99-7e17-4dd6-87d1-9eba21667e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:55.654496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:55.714557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1082/2185] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:32:55.725374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:32:55.727727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:55.768891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:32:55.787390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4bfd6-96c8-4acf-be6f-007ef0d9057b service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:32:55.791924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcda684-5e76-4719-86b5-6324b8cd9710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:32:55.792546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcda684-5e76-4719-86b5-6324b8cd9710 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:55.792881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcda684-5e76-4719-86b5-6324b8cd9710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:55.793425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcda684-5e76-4719-86b5-6324b8cd9710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:55.802756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcda684-5e76-4719-86b5-6324b8cd9710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:32:55.804373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcda684-5e76-4719-86b5-6324b8cd9710 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:32:55.814058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1104/2186] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:32:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:32:57.184918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:32:57.185477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:32:57.185890 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:32:57.208118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] MFA Rules not processed for user `57e1243ffa724189ba7d2b9277e10afd`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:32:57.209266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:32:57.217527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1083/2187] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:32:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:05.810752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=10 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:07.202014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=24 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:08.043352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670f69d-abff-48cb-a3a0-ce535114a904 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:08.043854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670f69d-abff-48cb-a3a0-ce535114a904 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:08.044152 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670f69d-abff-48cb-a3a0-ce535114a904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:08.066573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2670f69d-abff-48cb-a3a0-ce535114a904 None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:08.067911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2670f69d-abff-48cb-a3a0-ce535114a904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:08.167041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1105/2188] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:33:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:08.180472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:08.181960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:08.286834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cf755-2fb5-4735-93d8-6f3fbbcb6da9 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:08.293966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:08.294921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:08.295237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:08.295532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:08.309921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:08.314067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:08.332380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1084/2189] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:33:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:33:08.512164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75090a8d-beab-40c5-a40c-0674ea6400ab None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:08.512722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75090a8d-beab-40c5-a40c-0674ea6400ab None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:08.513207 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75090a8d-beab-40c5-a40c-0674ea6400ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:08.563306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-75090a8d-beab-40c5-a40c-0674ea6400ab None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:08.566154 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-75090a8d-beab-40c5-a40c-0674ea6400ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:08.576249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1106/2190] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:33:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:08.608785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:08.634166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6c5d6-8401-40c2-9bf0-506df7fee111 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:08.644095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:08.644720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:08.645054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:08.645334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:08.659723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:08.661906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:08.672989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1085/2191] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:33:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:33:09.916874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:09.918112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:09.918674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:09.952850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] MFA Rules not processed for user `48a91f6797cc4604ba2606e0549234d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:09.957513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:10.082566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1107/2192] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:11.419732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:11.440497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b21cc-7594-4494-9f88-bd4b73fdb026 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:11.446311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:11.446869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:11.447178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:11.447461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:11.449295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:11.600414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:11.602017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694800ab-7aec-46b5-bf09-ba9ed83c468d service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:11.617494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1086/2193] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:11.690351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:11.691929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:11.719489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be2cdd-a652-4622-a4c0-8dd53a3eb6db None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:11.731778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:11.732910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:11.733429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:11.733918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] PATH_INFO: `/v3/users/2d347f428d6e495eae08d35c350b6054` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:11.736824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d347f428d6e495eae08d35c350b6054)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:11.739137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:11.856150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d347f428d6e495eae08d35c350b6054'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:11.872184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d347f428d6e495eae08d35c350b6054'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:11.873318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d347f428d6e495eae08d35c350b6054'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:11.883287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d347f428d6e495eae08d35c350b6054'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:11.902578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d347f428d6e495eae08d35c350b6054'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:11.911417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1108/2194] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:33:11 2022] DELETE /identity/v3/users/2d347f428d6e495eae08d35c350b6054 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:11.917077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:11.917395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:11.917678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:11.951946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:11.953192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:12.108284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1087/2195] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:12.115197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:12.116775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:12.149515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:12.249012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c329e-f32a-4b41-8e22-376b00bfbb0e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:12.259445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe9dd5-e66b-46dc-92b6-bf41ab150664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:12.261055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe9dd5-e66b-46dc-92b6-bf41ab150664 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:12.262013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe9dd5-e66b-46dc-92b6-bf41ab150664 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:12.264728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe9dd5-e66b-46dc-92b6-bf41ab150664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:12.286998 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe9dd5-e66b-46dc-92b6-bf41ab150664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:12.290234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe9dd5-e66b-46dc-92b6-bf41ab150664 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:12.310949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1109/2196] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:12.833513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:12.835026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:12.856103 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db433ea-bbe4-4fda-b185-b5cbbbd73e9a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:12.881057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:12.882053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:12.882339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:12.883057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] PATH_INFO: `/v3/projects/e9bff7b0f0f542baa6c6ce5855cf212d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:12.884377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bff7b0f0f542baa6c6ce5855cf212d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:12.886651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:12.898057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:12.992212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bff7b0f0f542baa6c6ce5855cf212d', 'request_id': 'req-05e875a4-036b-4f22-83b5-118345fa0ee3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:13.001395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bff7b0f0f542baa6c6ce5855cf212d', 'request_id': 'req-05e875a4-036b-4f22-83b5-118345fa0ee3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:13.011664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bff7b0f0f542baa6c6ce5855cf212d', 'request_id': 'req-05e875a4-036b-4f22-83b5-118345fa0ee3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:13.035682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] The token cache is being invalidate because project e9bff7b0f0f542baa6c6ce5855cf212d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:33:13.036158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:13.037052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-05e875a4-036b-4f22-83b5-118345fa0ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:13.044217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1088/2197] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:33:12 2022] DELETE /identity/v3/projects/e9bff7b0f0f542baa6c6ce5855cf212d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:13.048965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.049670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.049806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.103864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:13.104439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:13.199481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1110/2198] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:13.207120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.207260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.207521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.240664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:13.242603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:13.253680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1089/2199] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:13.257989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:13.276049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66f5e-1344-4f31-b5c1-506be16d209e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:13.282141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:13.282906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.283332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.283721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.285553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:33:13.285996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:13.287907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:13.349163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1111/2200] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:33:13.353655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:13.372948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67306ed-bfab-449f-a81d-7f32e2cdf9f7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:13.384764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:13.386501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.387152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.387548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.388298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:13.394662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:13.478536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1090/2201] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:33:13 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:33:13.482243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:13.504446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1331802-a517-4bd0-b4c1-176b3d974117 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:13.688670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:13.689731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.690336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.690834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.692462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:13.697708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:13.767854 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1112/2202] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:33:13 2022] POST /identity/v3/users => generated 287 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:33:13.771967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:13.785530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba729ba-a5d9-40ae-ad48-842a746013da admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:13.801333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:13.803479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.805096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.805611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.808148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:13.810654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:13.830335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1091/2203] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:33:13.834929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:13.850574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a075-86f8-4540-974a-92abce13f6f9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:13.866294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0cd8b5-9798-417b-a81a-86aa2f05d9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:13.867448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cd8b5-9798-417b-a81a-86aa2f05d9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.867448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cd8b5-9798-417b-a81a-86aa2f05d9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.867448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0cd8b5-9798-417b-a81a-86aa2f05d9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.868475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cd8b5-9798-417b-a81a-86aa2f05d9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:13.870958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0cd8b5-9798-417b-a81a-86aa2f05d9c6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:13.884607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1113/2204] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:33:13.889633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:13.907666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15fa91-dd10-412e-a63c-a2e7f9e3c866 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:13.918425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:13.920010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.920706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.921198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] PATH_INFO: `/v3/projects/fb0c0d2823dd4a3fb35bb8c0cad4f410/users/c376f5bd19c840b7924093606dee6f4f/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:13.944239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb0c0d2823dd4a3fb35bb8c0cad4f410, user_id=c376f5bd19c840b7924093606dee6f4f, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:13.947680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:13.971018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1092/2205] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:33:13 2022] PUT /identity/v3/projects/fb0c0d2823dd4a3fb35bb8c0cad4f410/users/c376f5bd19c840b7924093606dee6f4f/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:13.976234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4a8d-8b6b-4c13-8cfc-421799042de6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:13.977499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4a8d-8b6b-4c13-8cfc-421799042de6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:13.978006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4a8d-8b6b-4c13-8cfc-421799042de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:14.016935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-8bfd4a8d-8b6b-4c13-8cfc-421799042de6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:14.020856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8bfd4a8d-8b6b-4c13-8cfc-421799042de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:14.057665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:14.172708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1114/2206] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:33:14.184792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:14.186716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:14.421147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3eafe-a8a0-47c6-b2f8-db588d26f0b0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:14.434721 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:14.435396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:14.435649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:14.435921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:14.460482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:14.462008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:14.501567 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1093/2207] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:23.884555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:23.885162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:23.885162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:23.966658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] MFA Rules not processed for user `c376f5bd19c840b7924093606dee6f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:23.968070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:24.132034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:24.152953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1115/2208] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:24.158333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c66734-8bf8-4e6c-8262-088b34f97979 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:24.164967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:24.168192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:24.171473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:24.171931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:24.172592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:24.174545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:24.243328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1094/2209] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:33:24 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:33:24.245876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:24.283725 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc230c-095c-44d3-86bc-ec50dae5bac1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:24.488214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:24.488992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:24.489336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:24.489610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:24.490261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:24.498252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:24.723432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1116/2210] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:33:24 2022] POST /identity/v3/users => generated 293 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:33:24.725777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:24.797045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15240b2a-50ec-4bfb-a4d5-bc6b23283659 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:24.834928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:24.837735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:24.838265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:24.838755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:24.841051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:24.844651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:24.869092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:24.902165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1095/2211] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:33:24.906837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:24.924626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6899e5-a642-4bb0-8865-54fd13d4802d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:24.932887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:24.933924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:24.934338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:24.934740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] PATH_INFO: `/v3/projects/77d7b40b3fe54711817a1d361b10a0c2/users/02887110cc77498e983edb258d4730c0/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:25.008684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d7b40b3fe54711817a1d361b10a0c2, user_id=02887110cc77498e983edb258d4730c0, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:25.011381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:25.040066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1117/2212] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:33:24 2022] PUT /identity/v3/projects/77d7b40b3fe54711817a1d361b10a0c2/users/02887110cc77498e983edb258d4730c0/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:33:25.042692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:25.075670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d36ef7-c3bd-46ba-bf6a-0a1388e112ea admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:25.293162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9980b7-31d8-4b8a-b8d9-1c005d58b3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:25.300612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9980b7-31d8-4b8a-b8d9-1c005d58b3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:25.301660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9980b7-31d8-4b8a-b8d9-1c005d58b3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:25.302681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9980b7-31d8-4b8a-b8d9-1c005d58b3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:25.304249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9980b7-31d8-4b8a-b8d9-1c005d58b3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:25.312069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9980b7-31d8-4b8a-b8d9-1c005d58b3b1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:25.366529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1096/2213] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:33:25.368743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:25.409826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d33dd-86a1-48c6-8a76-06884f2b8325 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:25.422035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:25.436588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:25.437479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:25.438307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] PATH_INFO: `/v3/projects/77d7b40b3fe54711817a1d361b10a0c2/users/02887110cc77498e983edb258d4730c0/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:25.459336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d7b40b3fe54711817a1d361b10a0c2, user_id=02887110cc77498e983edb258d4730c0, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:25.469355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:25.492709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1118/2214] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:33:25 2022] PUT /identity/v3/projects/77d7b40b3fe54711817a1d361b10a0c2/users/02887110cc77498e983edb258d4730c0/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:35.334726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=29 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:35.485044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:35.769224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bd395-63fc-4869-ae90-0259be2aa8e5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:35.770442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bd395-63fc-4869-ae90-0259be2aa8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:35.770888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bd395-63fc-4869-ae90-0259be2aa8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:35.865691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-894bd395-63fc-4869-ae90-0259be2aa8e5 None None] MFA Rules not processed for user `02887110cc77498e983edb258d4730c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:35.867041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-894bd395-63fc-4869-ae90-0259be2aa8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:36.076830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1097/2215] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:36.424259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:36.426474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:36.684363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22921bb-2b72-4b1b-ba26-11ef76531bc9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:36.694965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18554f3c-4c7d-4195-8a36-3e545ab4bef3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:36.696246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554f3c-4c7d-4195-8a36-3e545ab4bef3 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:36.696605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554f3c-4c7d-4195-8a36-3e545ab4bef3 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:36.696939 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554f3c-4c7d-4195-8a36-3e545ab4bef3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:36.789554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18554f3c-4c7d-4195-8a36-3e545ab4bef3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:36.791268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18554f3c-4c7d-4195-8a36-3e545ab4bef3 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:36.802411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1119/2216] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:38.680683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc54d-dce6-4bca-b3ea-12c95887e662 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:38.681297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc54d-dce6-4bca-b3ea-12c95887e662 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:38.681476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dc54d-dce6-4bca-b3ea-12c95887e662 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:38.682866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1098/2217] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:33:38.689204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3508189-75b1-4682-b054-727f7cf43231 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:38.689788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3508189-75b1-4682-b054-727f7cf43231 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:38.689788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3508189-75b1-4682-b054-727f7cf43231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:38.728557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d3508189-75b1-4682-b054-727f7cf43231 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:38.731110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d3508189-75b1-4682-b054-727f7cf43231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:38.779024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1120/2218] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:33:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:38.846764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc54d-dce6-4bca-b3ea-12c95887e662 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:38.856059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3b7dc54d-dce6-4bca-b3ea-12c95887e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:39.004179 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dc54d-dce6-4bca-b3ea-12c95887e662 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:39.008925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298d9c-2317-4f24-84d5-46ccbd4ec56b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:39.009556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298d9c-2317-4f24-84d5-46ccbd4ec56b service glance] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.009843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298d9c-2317-4f24-84d5-46ccbd4ec56b service glance] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.010136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298d9c-2317-4f24-84d5-46ccbd4ec56b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.030594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298d9c-2317-4f24-84d5-46ccbd4ec56b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:39.030594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d298d9c-2317-4f24-84d5-46ccbd4ec56b service glance] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:39.051675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1099/2219] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:39.166983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2017fea-513b-4802-82bf-ebd800755127 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.167810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2017fea-513b-4802-82bf-ebd800755127 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.168229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2017fea-513b-4802-82bf-ebd800755127 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.170020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1121/2220] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:33:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:33:39.175050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e3036-e3f7-495d-aa0b-40598ee08629 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.175719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e3036-e3f7-495d-aa0b-40598ee08629 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.176166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e3036-e3f7-495d-aa0b-40598ee08629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.204928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0e3e3036-e3f7-495d-aa0b-40598ee08629 None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:39.206544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0e3e3036-e3f7-495d-aa0b-40598ee08629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:39.309635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1100/2221] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:39.328769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2017fea-513b-4802-82bf-ebd800755127 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:39.349021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2017fea-513b-4802-82bf-ebd800755127 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:39.359647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877abd37-7a43-462f-a93e-afad43d040e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:39.360271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877abd37-7a43-462f-a93e-afad43d040e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.362377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877abd37-7a43-462f-a93e-afad43d040e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.363038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877abd37-7a43-462f-a93e-afad43d040e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.383576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877abd37-7a43-462f-a93e-afad43d040e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:39.387179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877abd37-7a43-462f-a93e-afad43d040e5 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:39.404069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1122/2222] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:39.566457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa628248-93ce-4939-befe-eda52bca276c None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.566764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa628248-93ce-4939-befe-eda52bca276c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.567184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa628248-93ce-4939-befe-eda52bca276c None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.568803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1101/2223] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:33:39.581506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7c688-a69a-4ffb-b47a-4e7c86677b3b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.581806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7c688-a69a-4ffb-b47a-4e7c86677b3b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.582142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7c688-a69a-4ffb-b47a-4e7c86677b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.596241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.596736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.597015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.636328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-5fa7c688-a69a-4ffb-b47a-4e7c86677b3b None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:39.636703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] MFA Rules not processed for user `c376f5bd19c840b7924093606dee6f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:39.638232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:39.639610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-5fa7c688-a69a-4ffb-b47a-4e7c86677b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:39.654689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1102/2224] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:39.660163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1123/2225] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:33:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:39.675758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:39.685573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e68f4-4036-41c3-84c7-5ff109529dd6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:39.690811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:39.691640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:39.692076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:39.696089 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:39.716277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:39.717811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:39.728333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:39.750934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1103/2226] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:33:40.111771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eb1d6-1925-48ef-839e-eef6a726966f None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:40.112308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eb1d6-1925-48ef-839e-eef6a726966f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:40.112308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0eb1d6-1925-48ef-839e-eef6a726966f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:40.160904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1f0eb1d6-1925-48ef-839e-eef6a726966f None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:40.162529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1f0eb1d6-1925-48ef-839e-eef6a726966f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:40.254435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1124/2227] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 14:33:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:33:40.280379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:40.281914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:40.515629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec22d4-dbc0-405d-8c43-ed55de52b3d5 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:40.530073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59691e9-dfb2-4fb0-bbb6-8a07b773cc1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:40.530811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59691e9-dfb2-4fb0-bbb6-8a07b773cc1b service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:40.533793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59691e9-dfb2-4fb0-bbb6-8a07b773cc1b service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:40.534191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59691e9-dfb2-4fb0-bbb6-8a07b773cc1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:40.558572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59691e9-dfb2-4fb0-bbb6-8a07b773cc1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:40.563772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59691e9-dfb2-4fb0-bbb6-8a07b773cc1b service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:40.594265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1104/2228] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:33:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:33:43.184141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ab75f-e95e-4f63-a79a-91837a90506b None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.185020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ab75f-e95e-4f63-a79a-91837a90506b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.185475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ab75f-e95e-4f63-a79a-91837a90506b None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:43.187935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1125/2229] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:33:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:33:43.205867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.206404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.206577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:43.233805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] MFA Rules not processed for user `90fd9b3148004cc2b8d14bced329b10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:43.235454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:43.249456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ab75f-e95e-4f63-a79a-91837a90506b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:43.253797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-232ab75f-e95e-4f63-a79a-91837a90506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:43.281727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:43.373132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1105/2230] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:43.385326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:43.386951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:43.388117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ab75f-e95e-4f63-a79a-91837a90506b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:43.393960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:43.394820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.395287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.395700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] PATH_INFO: `/v3/users/c29b21ee2cd341169a2c8263393c5db6` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:43.397059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29b21ee2cd341169a2c8263393c5db6)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:43.399855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:43.400509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb1c58-0231-44df-816f-23a2dbc86bef None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:43.406772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:43.407425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.407905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.408257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:43.437594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:43.440016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:43.460615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1106/2231] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:33:43.472743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29b21ee2cd341169a2c8263393c5db6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.482030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29b21ee2cd341169a2c8263393c5db6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.482923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29b21ee2cd341169a2c8263393c5db6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.494217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29b21ee2cd341169a2c8263393c5db6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.509239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-60c06040-6c0a-42af-9e2b-307c70962241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29b21ee2cd341169a2c8263393c5db6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.515782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1126/2232] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:33:43 2022] DELETE /identity/v3/users/c29b21ee2cd341169a2c8263393c5db6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:43.716501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:43.729649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc7b64-b9b0-4357-a0ab-8dbbd52cadf9 service placement] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:43.796977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:43.797615 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.797891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.798163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] PATH_INFO: `/v3/projects/d6dae806b3c14f3fa1d2a491214de9f8` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:43.799325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6dae806b3c14f3fa1d2a491214de9f8)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:43.801235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:43.816235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:43.855208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6dae806b3c14f3fa1d2a491214de9f8', 'request_id': 'req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.864608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6dae806b3c14f3fa1d2a491214de9f8', 'request_id': 'req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.877505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6dae806b3c14f3fa1d2a491214de9f8', 'request_id': 'req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.891965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] The token cache is being invalidate because project d6dae806b3c14f3fa1d2a491214de9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:33:43.892262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.892916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9fc2923c-d3d8-4cc6-b1c0-121e31ea7dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:33:43.894426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1107/2233] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:33:43 2022] DELETE /identity/v3/projects/d6dae806b3c14f3fa1d2a491214de9f8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:43.899936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.900228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.900512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:43.921774 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:43.923495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:43.983682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1127/2234] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:43.991274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:43.991582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:43.991795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.011848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:44.013071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:44.028934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1108/2235] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:44.030721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.041690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058d6bdf-3424-41be-9080-32c9a7f4c785 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.046851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.047584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.047987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.048331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.049060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:33:44.049460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.051186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.066793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1128/2236] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:33:44.068274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.078998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e795aa-a3d5-4cbf-aa72-33c48aa5e6d5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.083794 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.084400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.084680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.085598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.086234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.088216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.119144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1109/2237] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:33:44 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:33:44.123417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.133566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836a68-525f-4ba3-a031-80b45a08324e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.152960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:44.198808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.199521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.199731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.199982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.200616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.202544 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.227025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1129/2238] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:33:44 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:33:44.231357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.242524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396335d6-43ba-414f-ba60-3d9d1b78a913 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.247155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.247756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.248032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.248302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.248944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.250533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.262373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1110/2239] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:33:44.266261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.277093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e257f-73f5-4a20-a5f8-803ef3eb9cc9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.281821 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56f965-f0c0-4f14-a955-d6dbe3c7f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.282408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56f965-f0c0-4f14-a955-d6dbe3c7f140 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.282687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56f965-f0c0-4f14-a955-d6dbe3c7f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.282958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56f965-f0c0-4f14-a955-d6dbe3c7f140 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.283664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56f965-f0c0-4f14-a955-d6dbe3c7f140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.285229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56f965-f0c0-4f14-a955-d6dbe3c7f140 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.296422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1130/2240] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:33:44.300652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.310807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145abeb-63b1-423c-8a79-5d4588238359 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.315515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.316162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.316447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.316725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] PATH_INFO: `/v3/projects/792f06e014fb4c338759807f8705f4db/users/6d172745ab3b4f7aafffb99a7614a07a/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.332545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792f06e014fb4c338759807f8705f4db, user_id=6d172745ab3b4f7aafffb99a7614a07a, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.335992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.350481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1111/2241] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:33:44 2022] PUT /identity/v3/projects/792f06e014fb4c338759807f8705f4db/users/6d172745ab3b4f7aafffb99a7614a07a/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:33:44.360893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625acb1-c614-480e-983c-6d72535c3de7 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.361178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625acb1-c614-480e-983c-6d72535c3de7 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.361455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625acb1-c614-480e-983c-6d72535c3de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.389913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f625acb1-c614-480e-983c-6d72535c3de7 None None] MFA Rules not processed for user `6d172745ab3b4f7aafffb99a7614a07a`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:44.391163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f625acb1-c614-480e-983c-6d72535c3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:44.488161 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1131/2242] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:33:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:44.498679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:44.503295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:44.603642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf817d-5cf9-4547-98d7-53a0d61bddd4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:44.607742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:44.608239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:44.608516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:44.608786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:44.619907 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:44.621395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:44.632737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1112/2243] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:33:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:33:45.682644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:45.683212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:45.683212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:45.702575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:33:45.703843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:45.765859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1132/2244] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:33:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:33:45.789676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:33:45.790771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:33:45.839969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:45.873558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088522f4-1126-4ed7-8705-e9d46db6d751 service glance] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:33:45.879886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cb9bc-a982-415e-81f9-1fb9b752d41b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:33:45.880605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cb9bc-a982-415e-81f9-1fb9b752d41b service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:33:45.880863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cb9bc-a982-415e-81f9-1fb9b752d41b service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:33:45.881132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cb9bc-a982-415e-81f9-1fb9b752d41b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:33:45.891014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cb9bc-a982-415e-81f9-1fb9b752d41b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:33:45.893070 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cb9bc-a982-415e-81f9-1fb9b752d41b service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:33:45.904561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1113/2245] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:33:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:33:55.756123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:33:55.900166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=14 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:00.487956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:00.489721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:00.574342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ceb8e-98f3-4ebd-b02a-3c48a473f279 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:00.580680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:00.581296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:00.587742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:00.588241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:00.590467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:00.606806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:00.608449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:00.627489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1133/2246] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:34:10.614628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=32 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:23.900211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c0f0c-14e5-4583-b646-1e48122cdd19 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:23.900758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c0f0c-14e5-4583-b646-1e48122cdd19 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:23.900758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c0f0c-14e5-4583-b646-1e48122cdd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:23.925427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5c0c0f0c-14e5-4583-b646-1e48122cdd19 None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:23.927442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5c0c0f0c-14e5-4583-b646-1e48122cdd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:23.939429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1114/2247] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 14:34:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:23.956493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:23.976440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2da1f-92f2-46e4-bcbe-163d65f35f80 service placement] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:23.980157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:23.980765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:23.981045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:23.981406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:23.990606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:23.992217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:24.000569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1134/2248] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:34:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:34:24.256548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594fc9-6dcf-4809-b35a-e7d6e62052d0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:24.256872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594fc9-6dcf-4809-b35a-e7d6e62052d0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:24.257144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45594fc9-6dcf-4809-b35a-e7d6e62052d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:24.281119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-45594fc9-6dcf-4809-b35a-e7d6e62052d0 None None] MFA Rules not processed for user `7d373dfa8c474ade92e94dc73dd9926f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:24.282365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-45594fc9-6dcf-4809-b35a-e7d6e62052d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:24.342945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1115/2249] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:33.997631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=6 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:34.336264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=28 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:41.787374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:41.855359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb09b-bb88-46ed-bd88-846f26622c1a service swift] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:41.866524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:41.867418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:41.867771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:41.868103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] PATH_INFO: `/v3/users/6d172745ab3b4f7aafffb99a7614a07a` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:41.869500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d172745ab3b4f7aafffb99a7614a07a)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:41.871884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:41.928147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d172745ab3b4f7aafffb99a7614a07a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:41.936304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d172745ab3b4f7aafffb99a7614a07a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:41.937063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d172745ab3b4f7aafffb99a7614a07a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:41.944615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d172745ab3b4f7aafffb99a7614a07a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:41.957281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d172745ab3b4f7aafffb99a7614a07a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:41.964108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1135/2250] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:34:41 2022] DELETE /identity/v3/users/6d172745ab3b4f7aafffb99a7614a07a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:41.968554 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:41.969019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:41.969129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:41.989240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:41.990484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:42.072581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1116/2251] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:34:42.079322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.080779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:42.142615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7997794-a7d5-4b2d-b9ed-8887be5059c0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:42.147613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c6552-3ed1-4e4f-9b6a-1767eee50bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:42.148215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c6552-3ed1-4e4f-9b6a-1767eee50bae service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.148490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c6552-3ed1-4e4f-9b6a-1767eee50bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.148757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c6552-3ed1-4e4f-9b6a-1767eee50bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.158294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c6552-3ed1-4e4f-9b6a-1767eee50bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:42.159812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c6552-3ed1-4e4f-9b6a-1767eee50bae service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:42.170734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1136/2252] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:34:42.500924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.503031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:42.517658 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82343d-3a4f-414e-aad3-c194a3f8c185 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:42.522572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:42.523762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.523762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.523762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] PATH_INFO: `/v3/projects/792f06e014fb4c338759807f8705f4db` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.524899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=792f06e014fb4c338759807f8705f4db)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:42.526815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:42.584332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792f06e014fb4c338759807f8705f4db', 'request_id': 'req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:42.590186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792f06e014fb4c338759807f8705f4db', 'request_id': 'req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:42.596747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792f06e014fb4c338759807f8705f4db', 'request_id': 'req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:42.609732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] The token cache is being invalidate because project 792f06e014fb4c338759807f8705f4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:34:42.610022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:42.610655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-66b8dd84-e0fb-4bd3-b5d3-dca7f6e7bb5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:42.612206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1117/2253] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:34:42 2022] DELETE /identity/v3/projects/792f06e014fb4c338759807f8705f4db => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:42.617632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.618027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.618372 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.639143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:42.640350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:42.657709 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:42.706128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1137/2254] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:42.714716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.715002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.715435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.737933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:42.739875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:42.747910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1118/2255] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:42.752119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.762941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262efbc6-c958-45fe-9873-0ff98c0baf84 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:42.767530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:42.768271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.768554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.768831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.769489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:34:42.769793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:42.771491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:42.784415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1138/2256] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:34:42.789886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.799671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a37b0-b9e9-41ed-b5f3-3d487b74de0c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:42.806217 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:42.806843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.807274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.807691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.808561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:42.811063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:42.839779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1119/2257] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:34:42 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:34:42.843960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.854678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a914933-20d7-43eb-9605-21bd82cff1af admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:42.916895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:42.917811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.918188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.918559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.919326 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:42.921361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:42.947432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1139/2258] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:34:42 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:34:42.948638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.959060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d23466a-22aa-4ea6-92c9-6bf15fd43e6b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:42.964266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:42.964914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:42.965204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:42.965507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:42.966236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:42.967876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:42.982374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1120/2259] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:34:42.983959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:42.996530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09161bd-528d-4e8b-b4fe-08ad87cfb13a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:43.002221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621a571-f060-4169-bb9c-8973affbf1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:43.002978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621a571-f060-4169-bb9c-8973affbf1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.003383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621a571-f060-4169-bb9c-8973affbf1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.004048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621a571-f060-4169-bb9c-8973affbf1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.004780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621a571-f060-4169-bb9c-8973affbf1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:43.006838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621a571-f060-4169-bb9c-8973affbf1fb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:43.019668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1140/2260] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:34:43.024024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:43.034941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fff111-1f65-4628-a25c-86c40dc60632 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:43.041630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:43.042470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.042856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.043277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] PATH_INFO: `/v3/projects/fd3c25e64caa4942a1a4e2b969426677/users/8c6d39ae6b524be2afe4c704ab7c0d5f/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.066761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3c25e64caa4942a1a4e2b969426677, user_id=8c6d39ae6b524be2afe4c704ab7c0d5f, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:43.069539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:43.083841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1121/2261] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:34:43 2022] PUT /identity/v3/projects/fd3c25e64caa4942a1a4e2b969426677/users/8c6d39ae6b524be2afe4c704ab7c0d5f/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:34:43.094399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.094686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.094958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.126311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] MFA Rules not processed for user `8c6d39ae6b524be2afe4c704ab7c0d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:43.127851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:43.204914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1141/2262] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:43.217104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:43.218897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:43.242481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:43.284947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71d6c6-9530-4bc5-8167-e1ce08528fd1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:43.289271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b06d3-2cd4-461e-aa30-c221c344968a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:43.289884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b06d3-2cd4-461e-aa30-c221c344968a service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.290162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b06d3-2cd4-461e-aa30-c221c344968a service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.290436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b06d3-2cd4-461e-aa30-c221c344968a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.299535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b06d3-2cd4-461e-aa30-c221c344968a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:43.300985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b06d3-2cd4-461e-aa30-c221c344968a service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:43.310369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1122/2263] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:34:43.338151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:43.360655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:43.418994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e43ee-d898-4cbc-9dc9-f545fed16619 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:43.423629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:43.424261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.424540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.424813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] PATH_INFO: `/v3/users/8c6d39ae6b524be2afe4c704ab7c0d5f` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.426109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6d39ae6b524be2afe4c704ab7c0d5f)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:43.428105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:43.474549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6d39ae6b524be2afe4c704ab7c0d5f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:43.483790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6d39ae6b524be2afe4c704ab7c0d5f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:43.484426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6d39ae6b524be2afe4c704ab7c0d5f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:43.492356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6d39ae6b524be2afe4c704ab7c0d5f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:43.509108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6d39ae6b524be2afe4c704ab7c0d5f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:43.516525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1142/2264] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:34:43 2022] DELETE /identity/v3/users/8c6d39ae6b524be2afe4c704ab7c0d5f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:43.521687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.521976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.522252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.543360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:43.544984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:43.619136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1123/2265] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:43.628483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:43.630425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:43.696687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993effc-d380-4e3a-b2bc-823f35469d1a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:43.700895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6c498-2a97-4f80-8985-e8f6327bb4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:43.701660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6c498-2a97-4f80-8985-e8f6327bb4ac service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:43.702010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6c498-2a97-4f80-8985-e8f6327bb4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:43.702355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6c498-2a97-4f80-8985-e8f6327bb4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:43.714023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6c498-2a97-4f80-8985-e8f6327bb4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:43.715514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6c498-2a97-4f80-8985-e8f6327bb4ac service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:43.726082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1143/2266] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:34:44.004053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.006142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:44.018849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a31b8-2c5d-40bb-bc5e-aa058c8a85f7 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.023619 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.024256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.024534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.024806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] PATH_INFO: `/v3/projects/fd3c25e64caa4942a1a4e2b969426677` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.025969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3c25e64caa4942a1a4e2b969426677)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.027966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.079668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3c25e64caa4942a1a4e2b969426677', 'request_id': 'req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:44.089443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3c25e64caa4942a1a4e2b969426677', 'request_id': 'req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:44.095919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3c25e64caa4942a1a4e2b969426677', 'request_id': 'req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:44.109265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] The token cache is being invalidate because project fd3c25e64caa4942a1a4e2b969426677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:34:44.109537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:44.110310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68b82c56-e168-4fcf-90cb-16c2a25cb1e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:44.111939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1124/2267] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:34:44 2022] DELETE /identity/v3/projects/fd3c25e64caa4942a1a4e2b969426677 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:44.118620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.119908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.119908 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.139178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:44.140379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:44.175885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:44.204984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1144/2268] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:44.212639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.212918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.213190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.234590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:44.235842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:44.246133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1125/2269] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:44.252277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.262812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8684df7-835a-4250-83cc-788b88a845b0 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.269045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.269804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.270091 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.270358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.271017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:34:44.271366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.272987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.285214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1145/2270] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:34:44.290222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.300388 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33e8ec-094e-44dd-87fe-7227e33c7daf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.306599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.307333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.307683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.308031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.308767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.310807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.338289 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1126/2271] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:34:44 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:34:44.342667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.353859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef014b6-e586-4619-a8ae-aad1f55005a6 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.416194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.416814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.417142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.417417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.418075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.420058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.443947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1146/2272] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:34:44 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:34:44.447901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.458245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d289e-323b-4426-91cd-f4b0011a6648 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.462867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.463495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.463771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.464041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.464708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.466301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.478407 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1127/2273] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:34:44.483240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.494683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447e123-1838-49c2-bf0b-4f17fcec4a25 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.499835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669e1a-4bdf-42c6-916a-bea895d23a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.500457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669e1a-4bdf-42c6-916a-bea895d23a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.500766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669e1a-4bdf-42c6-916a-bea895d23a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.501072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669e1a-4bdf-42c6-916a-bea895d23a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.501811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669e1a-4bdf-42c6-916a-bea895d23a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.503441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669e1a-4bdf-42c6-916a-bea895d23a0c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.515512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1147/2274] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:34:44.519610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.529661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5458-ab71-489a-82d1-891c7c43d4e1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.534320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.534967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.535281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.535558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] PATH_INFO: `/v3/projects/da9e334820a64b17b12799b7b454f2a3/users/6f76402fcab34d28b6aff06cf3f34806/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.538331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:44.557945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9e334820a64b17b12799b7b454f2a3, user_id=6f76402fcab34d28b6aff06cf3f34806, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.563338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.577770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1128/2275] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:34:44 2022] PUT /identity/v3/projects/da9e334820a64b17b12799b7b454f2a3/users/6f76402fcab34d28b6aff06cf3f34806/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:34:44.649360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379e14c-d88f-46de-9914-9dfe65def627 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.649647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379e14c-d88f-46de-9914-9dfe65def627 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.649950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379e14c-d88f-46de-9914-9dfe65def627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.678848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-8379e14c-d88f-46de-9914-9dfe65def627 None None] MFA Rules not processed for user `6f76402fcab34d28b6aff06cf3f34806`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:44.680110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8379e14c-d88f-46de-9914-9dfe65def627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:44.746729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1148/2276] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:44.757481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:44.759433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:44.833844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eeb130-160a-42d1-98e3-88dade7d9a61 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:44.843133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:44.843687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:44.843968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:44.844241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:44.855276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:44.856734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:44.871792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1129/2277] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:34:50.684044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:50.684486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:50.684932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:50.707482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] MFA Rules not processed for user `57e1243ffa724189ba7d2b9277e10afd`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:50.709291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:50.745937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:50.775512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1149/2278] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:51.020803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:51.023004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:51.091848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5b48e-7851-4990-a20f-68b922b54a85 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:51.095692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:51.096309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:51.096593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:51.096869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:51.098640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:51.171454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:51.172972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c13138c-b19b-40bd-bb62-f4283b4c3b0a service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:51.194918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1130/2279] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:34:58.603526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:58.668602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd383b9e-e1fe-4748-8023-2efcc59c6196 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:58.673349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:58.674031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:58.674262 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:58.674536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] PATH_INFO: `/v3/users/6f76402fcab34d28b6aff06cf3f34806` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:58.675682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f76402fcab34d28b6aff06cf3f34806)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:58.677611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:58.724209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f76402fcab34d28b6aff06cf3f34806'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:58.731358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f76402fcab34d28b6aff06cf3f34806'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:58.731956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f76402fcab34d28b6aff06cf3f34806'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:58.738748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f76402fcab34d28b6aff06cf3f34806'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:58.750503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f76402fcab34d28b6aff06cf3f34806'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:58.756877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1150/2280] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:34:58 2022] DELETE /identity/v3/users/6f76402fcab34d28b6aff06cf3f34806 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:58.761835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:58.762367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:58.762466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:58.784335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:58.785563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:58.816842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:58.848562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1131/2281] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:58.858341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:58.860074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:58.934234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68331b59-c209-4937-a47b-8e471e39dc94 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:58.937996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de34e2-f3fd-41cb-af57-804dfbfc26d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:58.938617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de34e2-f3fd-41cb-af57-804dfbfc26d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:58.938902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de34e2-f3fd-41cb-af57-804dfbfc26d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:58.939205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de34e2-f3fd-41cb-af57-804dfbfc26d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:58.955292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de34e2-f3fd-41cb-af57-804dfbfc26d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:58.958221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de34e2-f3fd-41cb-af57-804dfbfc26d7 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:58.976143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1151/2282] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:34:59.328547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.330760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:59.351156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12508218-fe4f-4dd5-9224-d5fdbc239d10 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.356291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.357133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.357263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.357514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] PATH_INFO: `/v3/projects/da9e334820a64b17b12799b7b454f2a3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.358639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9e334820a64b17b12799b7b454f2a3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.360665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.414964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9e334820a64b17b12799b7b454f2a3', 'request_id': 'req-68463977-f1eb-461b-89c5-64cbc4b331ad'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:59.421676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9e334820a64b17b12799b7b454f2a3', 'request_id': 'req-68463977-f1eb-461b-89c5-64cbc4b331ad'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:59.429923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9e334820a64b17b12799b7b454f2a3', 'request_id': 'req-68463977-f1eb-461b-89c5-64cbc4b331ad'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:59.446519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] The token cache is being invalidate because project da9e334820a64b17b12799b7b454f2a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:34:59.446731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:59.447353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-68463977-f1eb-461b-89c5-64cbc4b331ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:34:59.448860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1132/2283] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:34:59 2022] DELETE /identity/v3/projects/da9e334820a64b17b12799b7b454f2a3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:59.454951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.455478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.456025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.468853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:34:59.485711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:59.486945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:59.556155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1152/2284] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:59.564655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.565111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.565400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.585708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:59.586950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:34:59.597314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1133/2285] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:34:59.602959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.615816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23a206-a15f-4751-836e-1fecd30b9401 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.621210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.622007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.622183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.622576 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.623341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:34:59.623646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.625268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.638164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1153/2286] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:34:59.642625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.656230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2c0bf-b7bc-4dac-adf8-d208d29c154e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.661413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.662714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.663007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.663437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.664355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.666290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.702940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1134/2287] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:34:59 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:34:59.706784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.719443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f447a-7ac3-466b-b68f-373c50ec9a3e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.792869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.793486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.793768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.794042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.794673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.796628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.823203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1154/2288] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:34:59 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:34:59.827466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.838795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab0442-c06b-4cfd-bf63-0205617452fd admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.843858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.844496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.844779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.845053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.845859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.847497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.862173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1135/2289] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:34:59.863773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.874621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e65cb-8cdf-461c-8c3a-29fd7f3a2940 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.879460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458703a-ce71-4f71-a08d-ff30b75dbef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.880071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458703a-ce71-4f71-a08d-ff30b75dbef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.880389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458703a-ce71-4f71-a08d-ff30b75dbef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.880658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458703a-ce71-4f71-a08d-ff30b75dbef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.881309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458703a-ce71-4f71-a08d-ff30b75dbef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.883139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458703a-ce71-4f71-a08d-ff30b75dbef8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.894509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1155/2290] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:34:59.899374 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:34:59.909543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa290-5b5e-4657-8e03-7167aa3ddb71 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:34:59.915245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:34:59.916248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.916460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.916793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] PATH_INFO: `/v3/projects/90d6ef85228f4676b9ac19e123e1462a/users/3cc610fe5d814027b01236949e145430/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.935031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90d6ef85228f4676b9ac19e123e1462a, user_id=3cc610fe5d814027b01236949e145430, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:34:59.937819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:34:59.953367 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1136/2291] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:34:59 2022] PUT /identity/v3/projects/90d6ef85228f4676b9ac19e123e1462a/users/3cc610fe5d814027b01236949e145430/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:34:59.959122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74e415-d306-48a9-9cf9-bc5d91a832c6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:34:59.959455 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74e415-d306-48a9-9cf9-bc5d91a832c6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:34:59.959730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74e415-d306-48a9-9cf9-bc5d91a832c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:34:59.979190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4d74e415-d306-48a9-9cf9-bc5d91a832c6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:34:59.980414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4d74e415-d306-48a9-9cf9-bc5d91a832c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:00.046099 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1156/2292] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:00.055651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:00.057399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:00.118489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd416f9-9296-46d4-b7a0-7647e32b1064 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:00.122144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:00.122744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:00.123022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:00.123313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:00.133184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:00.134614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:00.138628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:00.151261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1137/2293] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:35:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:35:04.118520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.119063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.119203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.140035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:04.151903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] MFA Rules not processed for user `3cc610fe5d814027b01236949e145430`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:04.153330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:04.226695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1157/2294] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:35:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:04.227381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:04.237232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73991a45-5dc3-404a-b107-2f9da97a6e72 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:04.242222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:04.242954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.243199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.243474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.244096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:04.246076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:04.275177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1138/2295] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:35:04 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:35:04.279302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:04.291165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcf625-ffd6-436b-bd4e-319df0b4ac55 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:04.353974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:04.354588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.354866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.355222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.355869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:04.357818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:04.380635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1158/2296] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:35:04 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:35:04.384893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:04.395665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbac6bc-1986-4c92-bfd5-352d68e24d01 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:04.402559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:04.403206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.403485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.403755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.404400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:04.405995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:04.418002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1139/2297] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:35:04.422230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:04.432814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881608b8-72bc-41cc-8e12-02d89a74d763 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:04.437728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:04.438384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.438674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.438978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] PATH_INFO: `/v3/projects/37963517458a4a8e8291a173befcdeaf/users/42859720d5fe4a609a4eca452a80bef2/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.456534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37963517458a4a8e8291a173befcdeaf, user_id=42859720d5fe4a609a4eca452a80bef2, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:04.459681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:04.475234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1159/2298] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:35:04 2022] PUT /identity/v3/projects/37963517458a4a8e8291a173befcdeaf/users/42859720d5fe4a609a4eca452a80bef2/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:35:04.478764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:04.489604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d0445-4b3b-44d1-a0b4-bdcfb12b8493 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:04.590880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04142c05-595d-4781-8d7b-73167bd92d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:04.591553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04142c05-595d-4781-8d7b-73167bd92d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.591836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04142c05-595d-4781-8d7b-73167bd92d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.592114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04142c05-595d-4781-8d7b-73167bd92d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.592994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04142c05-595d-4781-8d7b-73167bd92d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:04.594669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04142c05-595d-4781-8d7b-73167bd92d77 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:04.609202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1140/2299] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:35:04.613590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:04.641855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31317-27a3-4bfb-9dd3-9839172d8401 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:04.650744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:04.652442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:04.652950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:04.653417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] PATH_INFO: `/v3/projects/37963517458a4a8e8291a173befcdeaf/users/42859720d5fe4a609a4eca452a80bef2/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:04.657254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37963517458a4a8e8291a173befcdeaf, user_id=42859720d5fe4a609a4eca452a80bef2, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:04.660724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:04.687789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1160/2300] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:35:04 2022] PUT /identity/v3/projects/37963517458a4a8e8291a173befcdeaf/users/42859720d5fe4a609a4eca452a80bef2/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:35:09.404181 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc523239-da81-4091-8f27-3946ac1ba1d8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:09.404642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc523239-da81-4091-8f27-3946ac1ba1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:09.404912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc523239-da81-4091-8f27-3946ac1ba1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:09.436926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-cc523239-da81-4091-8f27-3946ac1ba1d8 None None] MFA Rules not processed for user `42859720d5fe4a609a4eca452a80bef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:09.438431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cc523239-da81-4091-8f27-3946ac1ba1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:09.533601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1141/2301] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:09.546285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:09.548118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:09.614035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c823be-d9ff-47d4-b22f-d1d5fde39a2f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:09.618159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:09.618762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:09.619200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:09.619470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:09.623813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:09.709080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:09.710839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:09.724641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1161/2302] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:35:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:35:09.868877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb3e2f-1967-4b09-9a7f-fcc97c145eca None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:09.869572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb3e2f-1967-4b09-9a7f-fcc97c145eca None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:09.869857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb3e2f-1967-4b09-9a7f-fcc97c145eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:09.891423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-e8eb3e2f-1967-4b09-9a7f-fcc97c145eca None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:09.892653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e8eb3e2f-1967-4b09-9a7f-fcc97c145eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:09.940947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:09.955710 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1142/2303] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:35:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:09.978543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:09.980151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:10.054794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc232152-4e79-4848-8b3a-ddd63ce8c969 service glance] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:10.059785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f47c04-1ac7-4712-9595-f6ce4ad4e384 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:10.060406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f47c04-1ac7-4712-9595-f6ce4ad4e384 service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:10.060686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f47c04-1ac7-4712-9595-f6ce4ad4e384 service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:10.060959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f47c04-1ac7-4712-9595-f6ce4ad4e384 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:10.070004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f47c04-1ac7-4712-9595-f6ce4ad4e384 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:10.071698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f47c04-1ac7-4712-9595-f6ce4ad4e384 service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:10.081031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1162/2304] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:35:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:35:19.040226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:19.040681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:19.040781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:19.059989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] MFA Rules not processed for user `3cc610fe5d814027b01236949e145430`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:19.061226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:19.068733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1143/2305] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:35:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:20.077846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=57 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:23.205079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f142bb-78b2-409a-bb5f-81f38efd1b5f None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:23.205682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f142bb-78b2-409a-bb5f-81f38efd1b5f None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:23.205682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f142bb-78b2-409a-bb5f-81f38efd1b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:23.227278 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-08f142bb-78b2-409a-bb5f-81f38efd1b5f None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:23.229450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-08f142bb-78b2-409a-bb5f-81f38efd1b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:23.237886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1163/2306] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:35:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:23.251975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:23.268116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3806bf0-76c4-42cc-8403-f17125fcc55c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:23.274657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:23.275398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:23.275906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:23.276254 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:23.293445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:23.295247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:23.307491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1144/2307] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:35:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:35:33.220888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:33.301341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=13 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:45.996543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:45.997117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:45.997117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:46.016655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] MFA Rules not processed for user `84d2f3c3c5fb4cceabf9a35f64a1551c`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:46.017873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:46.084194 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1164/2308] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:46.456725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:46.529776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785da749-8b39-4d0d-8665-685c0cde1067 service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:46.541099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:46.541627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:46.541925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:46.542195 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:46.544025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:46.627506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:46.629171 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868bbb13-b816-4989-8f1a-915bf77ff59b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:46.642032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1145/2309] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:35:46.778182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:46.791996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c23ac5-f27c-41c6-836b-b9c48e068545 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:46.798221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:46.798916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:46.799206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:46.799477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:46.802731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:46.817097 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:46.819140 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:46.831581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1165/2310] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:35:48.924045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f06ed-7524-455f-b696-324950a24eb7 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:48.924563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f06ed-7524-455f-b696-324950a24eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:48.924671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f06ed-7524-455f-b696-324950a24eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:48.950780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0c4f06ed-7524-455f-b696-324950a24eb7 None None] MFA Rules not processed for user `2495d3c63dc44049872afb908ca6133b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:48.952244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c4f06ed-7524-455f-b696-324950a24eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:49.029996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1146/2311] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:35:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:35:55.591226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:35:55.592665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:55.606276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab546ba-fe4a-4c23-a55f-e8ed8b720d77 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:35:55.611865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:35:55.612468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:55.612936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:55.613042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:55.615318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:55.626308 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:35:55.628234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:35:55.638734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1166/2312] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:35:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:35:55.875060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:35:55.875666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:35:55.875666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:35:55.895913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] MFA Rules not processed for user `48a91f6797cc4604ba2606e0549234d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:35:55.897140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:35:55.947457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:35:55.961600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1147/2313] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:35:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:36:01.998279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:02.010077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cbe6a-dbf5-4bca-9919-3fefced6e047 service nova] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:02.014129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:02.014826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:02.017482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:02.018180 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:02.028488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:02.030265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:02.042912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1167/2314] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:03.825353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:03.837049 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3817a77-02af-48fd-9da3-bc3e0258e066 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:03.841078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:03.841839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:03.842168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:03.842537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:03.845381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:03.865683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:03.867677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:03.882408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1148/2315] 10.222.0.50 () {60 vars in 1396 bytes} [Mon Aug 8 14:36:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:11.167671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:11.169255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:11.247530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2339bf5-3086-4973-8752-2d91af940bac service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:11.252316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:11.252945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:11.253407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:11.253516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] PATH_INFO: `/v3/users/57e1243ffa724189ba7d2b9277e10afd` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:11.255401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e1243ffa724189ba7d2b9277e10afd)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:11.257343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:11.306677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e1243ffa724189ba7d2b9277e10afd'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.317265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e1243ffa724189ba7d2b9277e10afd'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.317937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e1243ffa724189ba7d2b9277e10afd'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.327867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e1243ffa724189ba7d2b9277e10afd'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.333117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:36:11.343691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ab7db7af-6e24-48d4-8c9b-fd2f50c97975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e1243ffa724189ba7d2b9277e10afd'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.350533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1168/2316] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:36:11 2022] DELETE /identity/v3/users/57e1243ffa724189ba7d2b9277e10afd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:36:11.529922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:11.542224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b84f3-b6f9-4a8c-8e9b-6936c46f6992 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:11.601276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:11.601920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:11.602202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:11.602476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] PATH_INFO: `/v3/projects/693b1fece5b548c786a495d2dc91bd8a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:11.603657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] RBAC: Authorizing `identity:delete_project(project_id=693b1fece5b548c786a495d2dc91bd8a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:11.605644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:11.650521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693b1fece5b548c786a495d2dc91bd8a', 'request_id': 'req-fc11d9b2-406a-4aed-8ca0-525a776ddaae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.655914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693b1fece5b548c786a495d2dc91bd8a', 'request_id': 'req-fc11d9b2-406a-4aed-8ca0-525a776ddaae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.662414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693b1fece5b548c786a495d2dc91bd8a', 'request_id': 'req-fc11d9b2-406a-4aed-8ca0-525a776ddaae'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.675082 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] The token cache is being invalidate because project 693b1fece5b548c786a495d2dc91bd8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:36:11.675412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.676752 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:36:11.678234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1149/2317] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:36:11 2022] DELETE /identity/v3/projects/693b1fece5b548c786a495d2dc91bd8a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:36:12.240766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:12.241285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:12.241404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:12.260178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] MFA Rules not processed for user `7d373dfa8c474ade92e94dc73dd9926f`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:36:12.261375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:12.318693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1169/2318] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:36:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:36:17.589801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:17.591309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:17.657620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11d9b2-406a-4aed-8ca0-525a776ddaae admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:17.661246 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:17.661858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:17.662141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:17.662420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:17.664523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:17.674409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:17.675905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:17.684083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1150/2319] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:22.311204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=2,keystone:SELECT=30 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:36:25.750608 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:25.752150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:25.812658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d6cdb-32d7-40d0-9164-c9d2acaf8c15 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:25.816412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:25.817692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:25.817971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:25.818241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:25.820133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:25.879761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:25.881155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f86a5-580a-4b28-8c40-3d98c5aa4c1f service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:25.890391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1170/2320] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:27.680518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:36:29.598279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:29.599886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:29.699012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1ba4b-c812-4b48-972f-356099c2d1e4 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:29.705989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:29.706672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:29.706951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:29.707252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:29.709665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:29.724786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:29.726966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:29.742507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1151/2321] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:32.743192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:32.743721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:32.743721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:32.762753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] MFA Rules not processed for user `75b75be070a147fba4788b0f51d8d0fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:36:32.764007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:32.771456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1171/2322] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:36:33.076692 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:33.078102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:33.089508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a382d1d-d6a4-46ff-a97f-4edd2f2f5273 service nova] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:33.093209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:33.093808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:33.094087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:33.094360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:33.096069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:33.105376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:33.106824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:33.115114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1152/2323] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:33.421907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:33.423325 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:33.494426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a1685-2dc1-42e9-8fdb-82153821cf69 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:33.499554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:33.500164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:33.500452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:33.500717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:33.502430 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:33.514264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:33.515989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:33.527147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1172/2324] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:41.982674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:41.984164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:42.000718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7efc2-0e6e-4f23-aa51-b38fc5b7a9f2 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:42.006243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:42.006843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:42.007153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:42.007432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:42.009137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:42.021707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:42.023315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09cabf-19dd-44a1-9f93-c730a356b0d8 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:42.033767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1153/2325] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:43.521672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=83 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:36:45.328049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:36:45.340050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e38084-72d8-4cbf-bd14-02f02409a246 service placement] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:36:45.345589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:36:45.346185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:36:45.346464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:36:45.346737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:36:45.348504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:36:45.361303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:36:45.362922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:36:45.374218 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1173/2326] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:36:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:36:52.029342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=46 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:36:55.369784 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=6 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:19.788929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e64fd4-3bff-4a9b-8613-3b253c6b2ced None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:19.789401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e64fd4-3bff-4a9b-8613-3b253c6b2ced None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:19.789654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e64fd4-3bff-4a9b-8613-3b253c6b2ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:19.817643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f5e64fd4-3bff-4a9b-8613-3b253c6b2ced None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:19.818846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f5e64fd4-3bff-4a9b-8613-3b253c6b2ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:19.890377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1154/2327] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:37:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:19.940691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:19.942304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:20.033957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb0b1e-81f5-4611-800a-a04cb6d1e8a6 service nova] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:20.040344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:20.041077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:20.041505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:20.041851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:20.051775 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:20.053652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:20.067315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1174/2328] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:37:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:37:24.530054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:24.530583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:24.530583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:24.550456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] MFA Rules not processed for user `48a91f6797cc4604ba2606e0549234d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:24.551712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:24.618155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1155/2329] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:25.115743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:25.125730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a79527-ecf7-41a3-91ec-bebf20bb2ec3 service swift] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:25.130063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:25.130924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:25.131347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:25.131621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:25.133813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:25.149669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:25.151414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:25.164449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1175/2330] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:28.848867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:28.867916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5402a2-1eb4-434c-8db6-f03fd44173f4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:28.876322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:28.877397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:28.877397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:28.877741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:28.881776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:28.898696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:28.900861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a99a4-65f2-42f8-be34-55abaee95b81 service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:28.915739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1156/2331] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:35.157165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=38 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:38.740443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:38.749077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628821f6-03e5-4778-af42-258628f37b61 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:38.753209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:38.753845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:38.754118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:38.754489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:38.756827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:38.821800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:38.823258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e87f7f-8e46-4695-8804-1ac3c2f98e0b service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:38.832660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1176/2332] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:38.908497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=56 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:40.583076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd3449-26d1-4c66-a6bf-2c5bb9807b65 None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:40.584097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd3449-26d1-4c66-a6bf-2c5bb9807b65 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:40.584539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd3449-26d1-4c66-a6bf-2c5bb9807b65 None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:40.586322 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1157/2333] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:37:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:37:40.595361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c63e-8d7a-4b0a-8886-aee6ed325a5c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:40.595828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c63e-8d7a-4b0a-8886-aee6ed325a5c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:40.596059 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515c63e-8d7a-4b0a-8886-aee6ed325a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:40.626686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3515c63e-8d7a-4b0a-8886-aee6ed325a5c None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:40.628204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3515c63e-8d7a-4b0a-8886-aee6ed325a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:40.651484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1177/2334] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:37:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:40.659398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd3449-26d1-4c66-a6bf-2c5bb9807b65 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:40.662405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-34dd3449-26d1-4c66-a6bf-2c5bb9807b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:40.791905 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd3449-26d1-4c66-a6bf-2c5bb9807b65 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:40.795885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252fa15-c713-478e-967d-dc31ff36bcbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:40.796769 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252fa15-c713-478e-967d-dc31ff36bcbb service glance] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:40.797236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252fa15-c713-478e-967d-dc31ff36bcbb service glance] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:40.797808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252fa15-c713-478e-967d-dc31ff36bcbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:40.816827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252fa15-c713-478e-967d-dc31ff36bcbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:40.818576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252fa15-c713-478e-967d-dc31ff36bcbb service glance] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:40.848187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1158/2335] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:40.937499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418e46-e630-4924-a3bf-920a3d15c3c6 None None] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:40.937792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418e46-e630-4924-a3bf-920a3d15c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:40.938089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418e46-e630-4924-a3bf-920a3d15c3c6 None None] PATH_INFO: `/` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:40.939558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1178/2336] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:37:40.945352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52ae9-a218-44c8-846e-fd28a41054d4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:40.945641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52ae9-a218-44c8-846e-fd28a41054d4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:40.945931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52ae9-a218-44c8-846e-fd28a41054d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:40.967023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c2a52ae9-a218-44c8-846e-fd28a41054d4 None None] MFA Rules not processed for user `511ecf1b54294d088d4737c1841cad5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:40.968327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c2a52ae9-a218-44c8-846e-fd28a41054d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:40.982511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1159/2337] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:37:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:40.989452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418e46-e630-4924-a3bf-920a3d15c3c6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:41.002389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418e46-e630-4924-a3bf-920a3d15c3c6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:41.006332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3d2e5-9a38-47ad-8fad-9f685164809b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:41.007117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3d2e5-9a38-47ad-8fad-9f685164809b service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:41.007543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3d2e5-9a38-47ad-8fad-9f685164809b service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:41.007959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3d2e5-9a38-47ad-8fad-9f685164809b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:41.015966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3d2e5-9a38-47ad-8fad-9f685164809b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:41.017579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3d2e5-9a38-47ad-8fad-9f685164809b service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:41.030192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1179/2338] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:41.119614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea2493-f1ba-4be9-bba9-55c3d13b2def None None] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:41.119977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea2493-f1ba-4be9-bba9-55c3d13b2def None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:41.120469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea2493-f1ba-4be9-bba9-55c3d13b2def None None] PATH_INFO: `/` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:41.122288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1160/2339] 10.222.0.50 () {54 vars in 921 bytes} [Mon Aug 8 14:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 14:37:41.128023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:41.128387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:41.128738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:41.175437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] MFA Rules not processed for user `a587bc4f9082494ab0ab2b6ccbd517ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:41.176673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:41.205937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1180/2340] 10.222.0.50 () {58 vars in 1018 bytes} [Mon Aug 8 14:37:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:41.212660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea2493-f1ba-4be9-bba9-55c3d13b2def None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:41.224028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea2493-f1ba-4be9-bba9-55c3d13b2def None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:41.227676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:41.228400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:41.228765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:41.229102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:41.239678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:41.241164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:41.252363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1161/2341] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:42.924215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:42.925864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:42.941304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce06a2-6489-4c80-af4d-ab17ad146783 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:42.947997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:42.948875 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:42.949223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:42.950121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] PATH_INFO: `/v3/users/7d373dfa8c474ade92e94dc73dd9926f` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:42.951488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d373dfa8c474ade92e94dc73dd9926f)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:42.953940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.016793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d373dfa8c474ade92e94dc73dd9926f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.025223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d373dfa8c474ade92e94dc73dd9926f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.025881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d373dfa8c474ade92e94dc73dd9926f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.033858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d373dfa8c474ade92e94dc73dd9926f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.048501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-a38ff4c3-4718-40a3-8cbd-c35d46d6dda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d373dfa8c474ade92e94dc73dd9926f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.056389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1181/2342] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:37:42 2022] DELETE /identity/v3/users/7d373dfa8c474ade92e94dc73dd9926f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:37:43.320683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.350662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73baf4dd-afef-4122-a90b-1bbfca369ee2 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:43.409602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:43.410356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.410632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.410902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] PATH_INFO: `/v3/projects/6ecb83fbcae94dca82331ab87f028878` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.412624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ecb83fbcae94dca82331ab87f028878)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:43.414560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.462985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ecb83fbcae94dca82331ab87f028878', 'request_id': 'req-f6c4a465-373b-4064-b3a3-4444787a404f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.468946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ecb83fbcae94dca82331ab87f028878', 'request_id': 'req-f6c4a465-373b-4064-b3a3-4444787a404f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.476083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ecb83fbcae94dca82331ab87f028878', 'request_id': 'req-f6c4a465-373b-4064-b3a3-4444787a404f'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.490497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] The token cache is being invalidate because project 6ecb83fbcae94dca82331ab87f028878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:37:43.490791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.491479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f6c4a465-373b-4064-b3a3-4444787a404f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:43.493002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1162/2343] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:37:43 2022] DELETE /identity/v3/projects/6ecb83fbcae94dca82331ab87f028878 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:43.497956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.498240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.498512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.518613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:43.519909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:43.557529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:43.590674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1182/2344] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:37:43.598461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.598751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.599543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.621886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:43.623815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:43.631883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1163/2345] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:43.636189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.646386 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f28bd7-c71b-4007-8d10-0db293bd237c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:43.651041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:43.651673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.651952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.652224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.652906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:37:43.653208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:43.654810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.670831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1183/2346] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:37:43.672114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.682055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28a12d-b038-4e29-843c-a8a10a3b94bb None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:43.686775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:43.687398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.687671 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.687940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.688562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:43.690723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.695412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:43.734965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1164/2347] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:43 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:43.738787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.753693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e00f-fd57-4a59-97ff-a6abf17876e9 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:43.823467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:43.824272 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.824556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.824898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.825594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:43.827631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.854491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1184/2348] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:43 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:43.858553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.884174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42327ff-8cdd-4aa9-b3ef-2ac58946b164 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:43.892027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:43.893081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.893539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.893956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.894911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:43.897278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.913486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1165/2349] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:37:43.917725 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.932233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468791-10c0-48d1-8b47-348a5a56389d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:43.938396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66090e-45b8-4a67-b9be-a9b0e0f677ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:43.939261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66090e-45b8-4a67-b9be-a9b0e0f677ff admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:43.940034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66090e-45b8-4a67-b9be-a9b0e0f677ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:43.940370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66090e-45b8-4a67-b9be-a9b0e0f677ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:43.941780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66090e-45b8-4a67-b9be-a9b0e0f677ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:43.943657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66090e-45b8-4a67-b9be-a9b0e0f677ff admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:43.962254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1185/2350] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:43.966099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:43.986020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03610c-41e4-44c5-a9e7-e1aeb8a157e6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:44.014607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:44.015310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:44.015584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:44.016060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] PATH_INFO: `/v3/projects/99021fae40d44fe193cd522e77fc00be/users/494a65a27c774abba289f6d126087ff2/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:44.046719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99021fae40d44fe193cd522e77fc00be, user_id=494a65a27c774abba289f6d126087ff2, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:44.057953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:44.097109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1166/2351] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:43 2022] PUT /identity/v3/projects/99021fae40d44fe193cd522e77fc00be/users/494a65a27c774abba289f6d126087ff2/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:44.109766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:44.110303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:44.111037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:44.155028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] MFA Rules not processed for user `494a65a27c774abba289f6d126087ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:44.156762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:44.307974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1186/2352] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:44.380568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:44.382664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:44.477133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08fc77-7d94-4845-bf39-1ea594181e97 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:44.483219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6227214-30a3-4406-ac27-d536e32deebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a587bc4f9082494ab0ab2b6ccbd517ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:44.483889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6227214-30a3-4406-ac27-d536e32deebc service nova] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:44.484101 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6227214-30a3-4406-ac27-d536e32deebc service nova] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:44.484378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6227214-30a3-4406-ac27-d536e32deebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:44.494034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6227214-30a3-4406-ac27-d536e32deebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:44.495855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6227214-30a3-4406-ac27-d536e32deebc service nova] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:44.506094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1167/2353] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:44.774350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:44.842021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:44.922878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14363ab9-c357-4de8-b579-d45df16de047 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:44.939825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:44.940472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:44.940965 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:44.941533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] PATH_INFO: `/v3/users/494a65a27c774abba289f6d126087ff2` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:44.945653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] RBAC: Authorizing `identity:delete_user(user_id=494a65a27c774abba289f6d126087ff2)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:44.947945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:44.948935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:44.949228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:44.951260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:44.980666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:44.982552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:45.046053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '494a65a27c774abba289f6d126087ff2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:45.063416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '494a65a27c774abba289f6d126087ff2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:45.065188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '494a65a27c774abba289f6d126087ff2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:45.077316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '494a65a27c774abba289f6d126087ff2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:45.101224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1168/2354] 10.222.0.50 () {58 vars in 1009 bytes} [Mon Aug 8 14:37:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:45.104969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f35bd986-579c-4f77-85db-0af517fb472b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '494a65a27c774abba289f6d126087ff2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:45.113678 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:45.115758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:45.118066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1187/2355] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:37:44 2022] DELETE /identity/v3/users/494a65a27c774abba289f6d126087ff2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:45.122597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:45.122966 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:45.123284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:45.153411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:45.154662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:45.179150 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:45.224827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ac943-58b5-4d50-b886-e5f08a3d926d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:45.254221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1188/2356] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:37:45.262756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:45.264437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:45.302288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27749110-82bd-48b3-bcef-d5c00474514c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:45.303293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27749110-82bd-48b3-bcef-d5c00474514c service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:45.303293 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27749110-82bd-48b3-bcef-d5c00474514c service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:45.303513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27749110-82bd-48b3-bcef-d5c00474514c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:45.330500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af326150-3e19-41cf-823c-3e8b4a06938e None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:45.337330 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:45.341660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:45.342012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:45.342244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:45.362788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:45.364779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:45.378162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:45.387034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27749110-82bd-48b3-bcef-d5c00474514c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:45.389568 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27749110-82bd-48b3-bcef-d5c00474514c service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:45.391938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1189/2357] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:37:45.416075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1169/2358] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:37:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 14:37:46.185324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:46.186857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:46.223271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e3995-0b76-4c1b-81a3-504800c306e3 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:46.236220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:46.237335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:46.238583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:46.239039 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] PATH_INFO: `/v3/projects/99021fae40d44fe193cd522e77fc00be` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:46.244188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99021fae40d44fe193cd522e77fc00be)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:46.249938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:46.415146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99021fae40d44fe193cd522e77fc00be', 'request_id': 'req-80171123-51f5-4471-8efd-43a37cac46a7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:46.421533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99021fae40d44fe193cd522e77fc00be', 'request_id': 'req-80171123-51f5-4471-8efd-43a37cac46a7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:46.428620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99021fae40d44fe193cd522e77fc00be', 'request_id': 'req-80171123-51f5-4471-8efd-43a37cac46a7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:46.459328 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] The token cache is being invalidate because project 99021fae40d44fe193cd522e77fc00be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:37:46.459994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:46.460722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-80171123-51f5-4471-8efd-43a37cac46a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:46.465701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1190/2359] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:37:46 2022] DELETE /identity/v3/projects/99021fae40d44fe193cd522e77fc00be => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:46.468979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:46.469486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:46.469574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:46.511531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:46.512966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:46.657218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1170/2360] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:46.665638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:46.666284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:46.666834 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:46.715335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:46.716840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:46.744517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1191/2361] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:46.748606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:46.787646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d8296-7d1c-4598-869b-13d451ba0c7a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:46.797218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:46.797847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:46.798130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:46.798404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:46.799071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:37:46.799397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:46.801022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:46.827831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1171/2362] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:37:46.832016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:46.846582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af9d59-617e-4f09-8bf7-09efe29fa442 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:46.857565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:46.858728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:46.859389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:46.860979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:46.866676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:46.870606 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:46.916889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1192/2363] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:46 2022] POST /identity/v3/projects => generated 363 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:46.920961 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:46.939575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81be49-c84b-4127-9b33-14dcb551134e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:46.953323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:47.055257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.056808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.057306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.058137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.058977 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.061167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.098726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.111870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea85f8-ea29-491b-9367-8f42a4d03154 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.115491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1172/2364] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:46 2022] POST /identity/v3/users => generated 282 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:47.119861 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.120948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.120948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.121355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.122112 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.123911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.144377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.144534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1193/2365] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:47.169039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa4c8-d879-4930-a80c-5c414f3916f6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.176605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee7bd3-5b05-44fc-9333-5c47ee07248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.177899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee7bd3-5b05-44fc-9333-5c47ee07248d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.178722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee7bd3-5b05-44fc-9333-5c47ee07248d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.179120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee7bd3-5b05-44fc-9333-5c47ee07248d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.179931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee7bd3-5b05-44fc-9333-5c47ee07248d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.181918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee7bd3-5b05-44fc-9333-5c47ee07248d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.206150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.207331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1173/2366] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:37:47.223121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f8101-2542-475f-b212-cf5c2b85a4dc admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.230640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.232074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.232815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.233538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] PATH_INFO: `/v3/projects/231378652a634e519af8ca886aec157f/users/0c60d7f4b50a4997a87d11deb27207b2/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.261126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231378652a634e519af8ca886aec157f, user_id=0c60d7f4b50a4997a87d11deb27207b2, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.264098 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.298345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.298774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.299375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.302345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1194/2367] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:47 2022] PUT /identity/v3/projects/231378652a634e519af8ca886aec157f/users/0c60d7f4b50a4997a87d11deb27207b2/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:37:47.332993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] MFA Rules not processed for user `0c60d7f4b50a4997a87d11deb27207b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:47.336514 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:47.437037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1174/2368] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:47.440513 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.556099 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdcad0-a441-4a86-9b49-4c2c4d157cb8 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.561048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.561734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.562079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.562429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.563356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.565597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.627138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1195/2369] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:47 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:47.631980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.645454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e3efb-91ac-45c8-b1ba-24147b4a7a37 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.729838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.730486 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.730808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.731074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.731812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.733868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.768414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1175/2370] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:47 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:47.769702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.775977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:47.783516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465af20d-9def-4113-acad-ebefe13d6550 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.807629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.808315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.808666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.809024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.809949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.811838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.834253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1196/2371] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:47.835547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.851427 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02fc81-6eed-4f8b-b061-0994d0d777ac admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:47.895231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:47.897470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:47.904622 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:47.905284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] PATH_INFO: `/v3/projects/06134b8420a8421ca71c2d6c1198cd51/users/b2cf552be75d4a99a03cf39a59e4c0d8/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:47.911642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:47.942662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06134b8420a8421ca71c2d6c1198cd51, user_id=b2cf552be75d4a99a03cf39a59e4c0d8, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:47.945940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:47.972837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1176/2372] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:47 2022] PUT /identity/v3/projects/06134b8420a8421ca71c2d6c1198cd51/users/b2cf552be75d4a99a03cf39a59e4c0d8/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:47.977609 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:47.992906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52246c77-e482-4b5e-9dd1-3f6632a52ccc admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:48.130130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a9b53-147a-424e-ba00-4ae38c20de17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:48.133882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a9b53-147a-424e-ba00-4ae38c20de17 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:48.133882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a9b53-147a-424e-ba00-4ae38c20de17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:48.133882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a9b53-147a-424e-ba00-4ae38c20de17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:48.133882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a9b53-147a-424e-ba00-4ae38c20de17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:48.139687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a9b53-147a-424e-ba00-4ae38c20de17 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:48.162599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1197/2373] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:48.166625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:48.189623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f682c-22d5-4e0f-ad08-5860721d7053 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:48.203018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:48.204025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:48.204459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:48.204886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] PATH_INFO: `/v3/projects/06134b8420a8421ca71c2d6c1198cd51/users/b2cf552be75d4a99a03cf39a59e4c0d8/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:48.209498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06134b8420a8421ca71c2d6c1198cd51, user_id=b2cf552be75d4a99a03cf39a59e4c0d8, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:48.214817 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:48.244844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1177/2374] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:48 2022] PUT /identity/v3/projects/06134b8420a8421ca71c2d6c1198cd51/users/b2cf552be75d4a99a03cf39a59e4c0d8/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:48.256870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:48.257395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:48.257495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:48.342505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] MFA Rules not processed for user `b2cf552be75d4a99a03cf39a59e4c0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:48.347214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:48.581015 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1198/2375] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:48.595727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:48.597821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:48.683244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edddda-1080-439d-89db-dfc59cdd4f0b admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:48.687676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fa2de-2f51-4c58-9ee4-1bee862141fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:48.688466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fa2de-2f51-4c58-9ee4-1bee862141fa service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:48.688784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fa2de-2f51-4c58-9ee4-1bee862141fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:48.689158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fa2de-2f51-4c58-9ee4-1bee862141fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:48.700664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fa2de-2f51-4c58-9ee4-1bee862141fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:48.702617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fa2de-2f51-4c58-9ee4-1bee862141fa service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:48.732098 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1178/2376] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:49.531418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:49.673780 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f9ed2-0cc4-4636-ae5b-ca7a4385b359 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:49.683988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:49.684772 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:49.685151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:49.685451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] PATH_INFO: `/v3/users/0c60d7f4b50a4997a87d11deb27207b2` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:49.691964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c60d7f4b50a4997a87d11deb27207b2)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:49.694956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:49.740071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:49.825705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c60d7f4b50a4997a87d11deb27207b2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:49.841068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c60d7f4b50a4997a87d11deb27207b2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:49.842338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c60d7f4b50a4997a87d11deb27207b2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:49.856045 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c60d7f4b50a4997a87d11deb27207b2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:49.895690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c60d7f4b50a4997a87d11deb27207b2'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:49.906472 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1199/2377] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:37:49 2022] DELETE /identity/v3/users/0c60d7f4b50a4997a87d11deb27207b2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:49.912131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:49.912727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:49.912727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:49.955357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:49.955357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:50.158297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1179/2378] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:50.172105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:50.174013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:50.290973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115ff67-71c8-4b00-9537-5f3524d1d442 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:50.297851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:50.299835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:50.299985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:50.300229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:50.314384 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:50.316788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:50.349521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1200/2379] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:50.893706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:50.896334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:50.926137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d8842-09dc-44e6-aa94-5f376249e5b8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:50.943772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:50.944460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:50.944922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:50.945356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] PATH_INFO: `/v3/projects/231378652a634e519af8ca886aec157f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:50.946871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231378652a634e519af8ca886aec157f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:50.949278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:51.054695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231378652a634e519af8ca886aec157f', 'request_id': 'req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.064556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231378652a634e519af8ca886aec157f', 'request_id': 'req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.090039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231378652a634e519af8ca886aec157f', 'request_id': 'req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.108440 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:51.121400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] The token cache is being invalidate because project 231378652a634e519af8ca886aec157f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:37:51.122956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.123849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.125855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1180/2380] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:37:50 2022] DELETE /identity/v3/projects/231378652a634e519af8ca886aec157f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:51.130914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:51.146271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa46eaa-d098-48ce-a4af-55372a28de47 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:51.273570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:51.274226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:51.274536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:51.274791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] PATH_INFO: `/v3/users/b2cf552be75d4a99a03cf39a59e4c0d8` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:51.278934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2cf552be75d4a99a03cf39a59e4c0d8)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:51.280924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:51.373295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:51.390195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2cf552be75d4a99a03cf39a59e4c0d8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.409255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2cf552be75d4a99a03cf39a59e4c0d8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.410999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2cf552be75d4a99a03cf39a59e4c0d8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.424026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2cf552be75d4a99a03cf39a59e4c0d8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.477028 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5f21dce8-2a65-4615-9756-900439a64449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2cf552be75d4a99a03cf39a59e4c0d8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:51.497314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1201/2381] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:37:51 2022] DELETE /identity/v3/users/b2cf552be75d4a99a03cf39a59e4c0d8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:51.918542 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:51.925701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:51.949340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d1376-6cbd-4d6e-bc41-cc39eb1799d2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.051420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.053414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.054073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.054582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] PATH_INFO: `/v3/projects/06134b8420a8421ca71c2d6c1198cd51` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.056231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06134b8420a8421ca71c2d6c1198cd51)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.058323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.117068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06134b8420a8421ca71c2d6c1198cd51', 'request_id': 'req-c2078161-3ed4-4ec9-b39b-f8a73848e737'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:52.123746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06134b8420a8421ca71c2d6c1198cd51', 'request_id': 'req-c2078161-3ed4-4ec9-b39b-f8a73848e737'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:52.132184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06134b8420a8421ca71c2d6c1198cd51', 'request_id': 'req-c2078161-3ed4-4ec9-b39b-f8a73848e737'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:52.147431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] The token cache is being invalidate because project 06134b8420a8421ca71c2d6c1198cd51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:37:52.147729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:52.148349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c2078161-3ed4-4ec9-b39b-f8a73848e737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:52.149828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1181/2382] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:37:51 2022] DELETE /identity/v3/projects/06134b8420a8421ca71c2d6c1198cd51 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:52.155957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.156949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.157448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.180796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:52.182335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:52.274747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1202/2383] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:37:52.284795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.285265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.285545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.322790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:52.324418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:52.345756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1182/2384] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:52.350768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:52.363190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46736660-8226-45fa-a5ce-a99a4777ad5a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.382463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.383118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.383500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.383844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.384585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:37:52.384967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.386679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.401603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1203/2385] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:37:52.405919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:52.415630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff029663-98df-4ff6-8d5e-08fb1be4e2f6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.420302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.420906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.421182 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.421452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.422086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.424025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.456502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1183/2386] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:52 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:52.458491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:52.470196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0407534-7529-4426-956b-0833e195c95a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.546715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.548984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.548984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.548984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.548984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.551163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.619585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1204/2387] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:52 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:52.620722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:52.635393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21915237-1fb6-4253-b765-0b847d25d0e4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.643142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.643142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.643142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.643142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.643592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.645071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.661478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1184/2388] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:37:52.665677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:52.684230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262db1f-4c13-4701-9585-cfcefebfa656 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.692684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc069be-c7a5-43b4-8f0a-4c838110cba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.693315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc069be-c7a5-43b4-8f0a-4c838110cba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.693593 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc069be-c7a5-43b4-8f0a-4c838110cba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.694086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc069be-c7a5-43b4-8f0a-4c838110cba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.694738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc069be-c7a5-43b4-8f0a-4c838110cba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.696395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc069be-c7a5-43b4-8f0a-4c838110cba9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.723294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1205/2389] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:52.724500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:52.742979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd005c-381a-40b6-8454-449a573b1b61 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:52.748199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:52.748818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.749104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.749384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] PATH_INFO: `/v3/projects/df30604324524a2ca7b82c3673b21012/users/af1a889f86e642898f02ce207fff1005/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.765901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=df30604324524a2ca7b82c3673b21012, user_id=af1a889f86e642898f02ce207fff1005, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:52.768729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:52.782413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1185/2390] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:52 2022] PUT /identity/v3/projects/df30604324524a2ca7b82c3673b21012/users/af1a889f86e642898f02ce207fff1005/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:52.794035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:52.794320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:52.794595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:52.824160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] MFA Rules not processed for user `af1a889f86e642898f02ce207fff1005`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:52.825996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:52.852838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:52.961440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1206/2391] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:52.963580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.006761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:53.046738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc51d-6e56-4fd0-b05d-e92f07980fbc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.051510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.052637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.052915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.053183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.053809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.055756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.081989 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1186/2392] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:52 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:53.086151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.096928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da055533-70e6-49ba-8b95-dace175f5fa2 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.164957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.165652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.165783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.166079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.166760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.168964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.190963 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1207/2393] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:53 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:53.194997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.205709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4edec8-ef49-430d-8ca4-0210cfe100a0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.210413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.211019 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.211321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.211593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.212241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.213843 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.225537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1187/2394] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:53.230319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.243403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29eeefd-9d2b-45f5-bfae-dfc67fd974cd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.248927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.249907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.250187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.250459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] PATH_INFO: `/v3/projects/988663f2f95e494a934412a1e9f4ee45/users/167f4eca5bbf4d848593449b2c0ce116/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.267564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=988663f2f95e494a934412a1e9f4ee45, user_id=167f4eca5bbf4d848593449b2c0ce116, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.270718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.288696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1208/2395] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:53 2022] PUT /identity/v3/projects/988663f2f95e494a934412a1e9f4ee45/users/167f4eca5bbf4d848593449b2c0ce116/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:53.290314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.310964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e8a8c-ec65-4dec-a50e-ba5850055549 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.371166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f8754b-b53c-4a3f-80e3-325f214b118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.371781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8754b-b53c-4a3f-80e3-325f214b118a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.372076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8754b-b53c-4a3f-80e3-325f214b118a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.372345 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f8754b-b53c-4a3f-80e3-325f214b118a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.373009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8754b-b53c-4a3f-80e3-325f214b118a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.374587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f8754b-b53c-4a3f-80e3-325f214b118a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.384441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1188/2396] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:53.401364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.412541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe569-4c46-4871-b9cd-e9bf7e23ba3b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.418201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.418878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.419266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.419679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] PATH_INFO: `/v3/projects/988663f2f95e494a934412a1e9f4ee45/users/167f4eca5bbf4d848593449b2c0ce116/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.422090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] RBAC: Authorizing `identity:create_grant(project_id=988663f2f95e494a934412a1e9f4ee45, user_id=167f4eca5bbf4d848593449b2c0ce116, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.425634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.443134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1209/2397] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:53 2022] PUT /identity/v3/projects/988663f2f95e494a934412a1e9f4ee45/users/167f4eca5bbf4d848593449b2c0ce116/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:53.449914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.450205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.450482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.487139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] MFA Rules not processed for user `167f4eca5bbf4d848593449b2c0ce116`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:53.488446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:53.556141 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1189/2398] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:53.566811 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:53.568531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:53.630577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299548e0-610a-4b51-a684-a4568171382b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:53.634242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0de53-c2c7-46af-9e3c-aca0d5f165fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:53.634841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0de53-c2c7-46af-9e3c-aca0d5f165fb service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:53.635146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0de53-c2c7-46af-9e3c-aca0d5f165fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:53.635420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0de53-c2c7-46af-9e3c-aca0d5f165fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:53.644146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0de53-c2c7-46af-9e3c-aca0d5f165fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:53.646074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0de53-c2c7-46af-9e3c-aca0d5f165fb service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:53.657580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1210/2399] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:55.249970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:55.278164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:55.324495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10b177-c3d7-4424-b09c-e60d0a3e1ab6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:55.329882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:55.330521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:55.330801 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:55.331074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] PATH_INFO: `/v3/users/af1a889f86e642898f02ce207fff1005` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:55.332234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1a889f86e642898f02ce207fff1005)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:55.335140 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:55.388967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1a889f86e642898f02ce207fff1005'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.396504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1a889f86e642898f02ce207fff1005'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.405557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1a889f86e642898f02ce207fff1005'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.406426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1a889f86e642898f02ce207fff1005'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.422204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1a889f86e642898f02ce207fff1005'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.431163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1190/2400] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:37:55 2022] DELETE /identity/v3/users/af1a889f86e642898f02ce207fff1005 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:55.433997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4005c-e061-44be-866a-0e712575d359 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:55.434865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4005c-e061-44be-866a-0e712575d359 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:55.435170 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4005c-e061-44be-866a-0e712575d359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:55.444486 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:55.456877 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-17f4005c-e061-44be-866a-0e712575d359 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:55.458840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-17f4005c-e061-44be-866a-0e712575d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:55.536785 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1211/2401] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:55.546885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:55.548307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:55.605640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacbbfb-118e-4326-9584-83073befed3d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:55.609277 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:55.609877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:55.610156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:55.610423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:55.619191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:55.620603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:55.630306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1191/2402] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:55.901930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4005c-e061-44be-866a-0e712575d359 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:55.903392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-17f4005c-e061-44be-866a-0e712575d359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:55.916823 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4005c-e061-44be-866a-0e712575d359 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:55.921553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:55.922182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:55.922463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:55.922735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] PATH_INFO: `/v3/projects/df30604324524a2ca7b82c3673b21012` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:55.923928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=df30604324524a2ca7b82c3673b21012)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:55.925902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:55.972331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df30604324524a2ca7b82c3673b21012', 'request_id': 'req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.977807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df30604324524a2ca7b82c3673b21012', 'request_id': 'req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.983878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df30604324524a2ca7b82c3673b21012', 'request_id': 'req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.996356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] The token cache is being invalidate because project df30604324524a2ca7b82c3673b21012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:37:55.996648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.997260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:55.998713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1212/2403] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:37:55 2022] DELETE /identity/v3/projects/df30604324524a2ca7b82c3673b21012 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:56.002901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:56.012998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f2dea4-7592-444e-a19e-ee99c6eeb361 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:56.052952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:56.070333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:56.070965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.071334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.071611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] PATH_INFO: `/v3/users/167f4eca5bbf4d848593449b2c0ce116` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.072740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] RBAC: Authorizing `identity:delete_user(user_id=167f4eca5bbf4d848593449b2c0ce116)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:56.074758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:56.119469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167f4eca5bbf4d848593449b2c0ce116'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.126379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167f4eca5bbf4d848593449b2c0ce116'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.133468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167f4eca5bbf4d848593449b2c0ce116'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.134097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167f4eca5bbf4d848593449b2c0ce116'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.145531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-da6e1d5f-1c54-471c-bbed-7798da2b525b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167f4eca5bbf4d848593449b2c0ce116'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.151170 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1192/2404] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:37:56 2022] DELETE /identity/v3/users/167f4eca5bbf4d848593449b2c0ce116 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:56.422186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:56.423670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:56.437719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe91b8b-50eb-408a-ae12-12a50f2e0adf admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:56.505527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:56.506084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.506366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.506636 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] PATH_INFO: `/v3/projects/988663f2f95e494a934412a1e9f4ee45` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.507796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=988663f2f95e494a934412a1e9f4ee45)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:56.509760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:56.555505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:56.592390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988663f2f95e494a934412a1e9f4ee45', 'request_id': 'req-81135bb4-d4b0-4376-a8f9-90363d12eb33'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.605424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988663f2f95e494a934412a1e9f4ee45', 'request_id': 'req-81135bb4-d4b0-4376-a8f9-90363d12eb33'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.616009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988663f2f95e494a934412a1e9f4ee45', 'request_id': 'req-81135bb4-d4b0-4376-a8f9-90363d12eb33'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.634247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] The token cache is being invalidate because project 988663f2f95e494a934412a1e9f4ee45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:37:56.634547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.635230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-81135bb4-d4b0-4376-a8f9-90363d12eb33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:37:56.636717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1213/2405] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:37:56 2022] DELETE /identity/v3/projects/988663f2f95e494a934412a1e9f4ee45 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:56.641791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.642300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.642410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.665078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:56.666445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:56.733441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1193/2406] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:56.741518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.741801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.742071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.761276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:56.762394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:56.771307 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1214/2407] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:56.775304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:56.784929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc409-9013-4988-ad27-3fa7d6e8db5b None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:56.790306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:56.790898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.791220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.791495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.792137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:37:56.792443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:56.794041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:56.805506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1194/2408] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:37:56.809967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:56.819599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c454de-cdd1-4e5a-95d7-0856a992f180 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:56.824277 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:56.824923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.825204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.825475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.826123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:56.828065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:56.855484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1215/2409] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:56 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:56.859703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:56.869953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5302-d1e1-4dc9-a8fb-c4009cbfdd80 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:56.914811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:56.928144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:56.928749 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.929039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.929321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.929934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:56.931899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:56.953397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1195/2410] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:56 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:56.957645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:56.968597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70473f3e-7ca5-43ce-b932-166e1f10cd2d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:56.973238 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:56.973818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:56.974092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:56.974357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:56.975001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:56.976722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:56.986477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1216/2411] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:56.990260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.000130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abcb5e-f7d2-4700-be93-bb590b414ba0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.005295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72828708-a446-4d6a-86e6-937e93a8325d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.005883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72828708-a446-4d6a-86e6-937e93a8325d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.006165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72828708-a446-4d6a-86e6-937e93a8325d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.006437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72828708-a446-4d6a-86e6-937e93a8325d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.007075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72828708-a446-4d6a-86e6-937e93a8325d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.008684 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72828708-a446-4d6a-86e6-937e93a8325d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.019118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1196/2412] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:57.023572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.033257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7984838-00ff-4348-876e-1f41767895ab admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.038364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.039005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.039313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.039590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] PATH_INFO: `/v3/projects/40a9363713954bf2b457bf85c68c88c0/users/1e5a8afa049a418cb8664d1500e31523/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.053692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a9363713954bf2b457bf85c68c88c0, user_id=1e5a8afa049a418cb8664d1500e31523, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.056360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.069094 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1217/2413] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:57 2022] PUT /identity/v3/projects/40a9363713954bf2b457bf85c68c88c0/users/1e5a8afa049a418cb8664d1500e31523/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:57.080422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.080741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.081018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.108540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] MFA Rules not processed for user `1e5a8afa049a418cb8664d1500e31523`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:57.109760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:57.171726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1197/2414] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:37:57.176234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.241857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8b699-302b-43cd-88a5-25e11a9d24a6 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.247413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.248171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.248547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.249030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.249542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.251698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.278685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1218/2415] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:37:57 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:57.282870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.293937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3d7f3-4278-47c2-99be-e31a070b2ae4 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.348664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.349265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.349546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.349822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.350441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.352393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.374133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1198/2416] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:37:57 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:37:57.378118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.388916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c75c58-baf1-42a7-901a-99ff1df490d8 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.393522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.394086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.394366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.394637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.395300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.396895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.407518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1219/2417] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:37:57.412063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.423056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ef703-b16d-4ed6-9103-dec1e3f4c00c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.428243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.428911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.429173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.429478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] PATH_INFO: `/v3/projects/6a1f01016e7b443497698960df2acb05/users/927b3f316b8f4adba8fdd35875345571/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.444389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1f01016e7b443497698960df2acb05, user_id=927b3f316b8f4adba8fdd35875345571, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.447036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.459952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1199/2418] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:57 2022] PUT /identity/v3/projects/6a1f01016e7b443497698960df2acb05/users/927b3f316b8f4adba8fdd35875345571/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:37:57.463476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.473717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b07aa7-a609-4970-81b9-d6e09efeea82 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.499491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:57.534543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51615b8-bbe0-4e5a-8ce1-5625b4c95c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.535070 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51615b8-bbe0-4e5a-8ce1-5625b4c95c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.535378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51615b8-bbe0-4e5a-8ce1-5625b4c95c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.535660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51615b8-bbe0-4e5a-8ce1-5625b4c95c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.536584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51615b8-bbe0-4e5a-8ce1-5625b4c95c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.538163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51615b8-bbe0-4e5a-8ce1-5625b4c95c8c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.548373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1220/2419] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:37:57.552994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.562929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db7874-75ce-4244-9222-0503f0e76a85 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.568352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.568995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.569276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.569550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] PATH_INFO: `/v3/projects/6a1f01016e7b443497698960df2acb05/users/927b3f316b8f4adba8fdd35875345571/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.571450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1f01016e7b443497698960df2acb05, user_id=927b3f316b8f4adba8fdd35875345571, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.574298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.587863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1200/2420] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:37:57 2022] PUT /identity/v3/projects/6a1f01016e7b443497698960df2acb05/users/927b3f316b8f4adba8fdd35875345571/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:37:57.596222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.596535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.596810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.623887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] MFA Rules not processed for user `927b3f316b8f4adba8fdd35875345571`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:37:57.625065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:57.696260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1221/2421] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:37:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:37:57.706657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.708451 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:37:57.719482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:37:57.773935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682447-d39f-4e30-8455-2562c42d1ec3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.778117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.778885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.779373 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.779848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:57.790686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:57.792268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:57.804131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1201/2422] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:37:57.917728 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:37:57.927976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3475b0c7-fdf3-4ad1-9f8c-d657d73209ba None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:37:57.931554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91986f-d99a-42e5-a89d-689b54e63e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:37:57.932158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91986f-d99a-42e5-a89d-689b54e63e3e service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:37:57.932462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91986f-d99a-42e5-a89d-689b54e63e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:37:57.932741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91986f-d99a-42e5-a89d-689b54e63e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:37:58.002450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91986f-d99a-42e5-a89d-689b54e63e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:37:58.004099 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91986f-d99a-42e5-a89d-689b54e63e3e service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:37:58.016153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1222/2423] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:37:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:00.609984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:00.682130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226cdf8-2e7e-4723-9f42-f6c6b6851c22 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:00.686698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:00.687690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:00.687972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:00.688251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] PATH_INFO: `/v3/users/1e5a8afa049a418cb8664d1500e31523` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:00.689389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5a8afa049a418cb8664d1500e31523)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:00.691357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:00.754392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5a8afa049a418cb8664d1500e31523'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:00.764539 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5a8afa049a418cb8664d1500e31523'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:00.777107 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5a8afa049a418cb8664d1500e31523'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:00.777818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5a8afa049a418cb8664d1500e31523'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:00.793928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5a8afa049a418cb8664d1500e31523'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:00.803325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1202/2424] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:00 2022] DELETE /identity/v3/users/1e5a8afa049a418cb8664d1500e31523 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:00.809237 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547957da-a971-455f-80a3-3d9914b88490 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:00.809713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547957da-a971-455f-80a3-3d9914b88490 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:00.813222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547957da-a971-455f-80a3-3d9914b88490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:00.835617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-547957da-a971-455f-80a3-3d9914b88490 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:00.836907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-547957da-a971-455f-80a3-3d9914b88490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:00.884628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:00.900595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1223/2425] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:00.910494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:00.912881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:00.947264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:00.974478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4076973-9d87-4ddf-ae58-1c84daaef1c1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:00.978128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:00.978729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:00.979168 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:00.979458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:00.989490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:00.990908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:01.000718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1203/2426] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:01.303843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547957da-a971-455f-80a3-3d9914b88490 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:01.305347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-547957da-a971-455f-80a3-3d9914b88490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:01.320206 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547957da-a971-455f-80a3-3d9914b88490 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:01.325633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:01.326295 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:01.326531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:01.326800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] PATH_INFO: `/v3/projects/40a9363713954bf2b457bf85c68c88c0` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:01.327950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a9363713954bf2b457bf85c68c88c0)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:01.329905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:01.377015 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a9363713954bf2b457bf85c68c88c0', 'request_id': 'req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.383479 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a9363713954bf2b457bf85c68c88c0', 'request_id': 'req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.389589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a9363713954bf2b457bf85c68c88c0', 'request_id': 'req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.406766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] The token cache is being invalidate because project 40a9363713954bf2b457bf85c68c88c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:01.406766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.407426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.414781 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1224/2427] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:01 2022] DELETE /identity/v3/projects/40a9363713954bf2b457bf85c68c88c0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:01.418729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:01.428536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe13c5-a7ff-45f3-a768-c4b50839c5eb service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:01.492248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:01.492934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:01.493208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:01.493476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] PATH_INFO: `/v3/users/927b3f316b8f4adba8fdd35875345571` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:01.494719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=927b3f316b8f4adba8fdd35875345571)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:01.496676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:01.543781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927b3f316b8f4adba8fdd35875345571'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.550672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927b3f316b8f4adba8fdd35875345571'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.558252 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927b3f316b8f4adba8fdd35875345571'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.558898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927b3f316b8f4adba8fdd35875345571'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.572025 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-02e6c11a-df8d-4fbf-a617-a2645ac48b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927b3f316b8f4adba8fdd35875345571'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.578559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1204/2428] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:01 2022] DELETE /identity/v3/users/927b3f316b8f4adba8fdd35875345571 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:01.855043 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:01.856585 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:01.870360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc6c9b-73bb-4bf1-b425-ada1c2956031 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:01.927288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:01.927910 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:01.928182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:01.928448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] PATH_INFO: `/v3/projects/6a1f01016e7b443497698960df2acb05` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:01.929538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1f01016e7b443497698960df2acb05)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:01.931471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:01.975678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1f01016e7b443497698960df2acb05', 'request_id': 'req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.981077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1f01016e7b443497698960df2acb05', 'request_id': 'req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.987029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1f01016e7b443497698960df2acb05', 'request_id': 'req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:01.999449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] The token cache is being invalidate because project 6a1f01016e7b443497698960df2acb05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:01.999739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:02.000356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0e5cba7d-d6c8-48c8-96e0-4d7572ba69f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:02.001796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1225/2429] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:01 2022] DELETE /identity/v3/projects/6a1f01016e7b443497698960df2acb05 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:02.007340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.007805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.007900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.026999 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:02.028166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:02.074531 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:02.088065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1205/2430] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:02.095815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.096125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.096401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.103927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:02.117904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:02.119106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:02.126783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1226/2431] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:02.131573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.141083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc02a6e-9c83-4a0d-9978-30409ad12eb3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.145715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.146309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.146591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.146864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.147543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:02.147847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.149428 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.161558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1206/2432] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:02.165641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.174551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476a6ef-cbab-4889-a0e6-a7fe17b19311 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.179766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.180345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.180622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.180888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.181495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.183397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.209306 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1227/2433] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:38:02.213249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.222901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fa042-1003-4d23-814e-49c84243c686 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.278109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.278653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.278931 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.279235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.279860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.281805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.305487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1207/2434] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:02.309435 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.319542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1631f-b32e-43f1-87c0-856e7e288502 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.324126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.324723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.325001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.325270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.325912 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.327531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.337938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1228/2435] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:38:02.341727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.352489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abb451-5153-42f1-83ef-ed5fd5a89e27 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.357120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20b53e-592a-4e9d-9ae9-56cde4c7eff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.357719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b53e-592a-4e9d-9ae9-56cde4c7eff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.357998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b53e-592a-4e9d-9ae9-56cde4c7eff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.358268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20b53e-592a-4e9d-9ae9-56cde4c7eff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.358911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b53e-592a-4e9d-9ae9-56cde4c7eff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.360545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20b53e-592a-4e9d-9ae9-56cde4c7eff8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.371902 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1208/2436] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:02.376189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.386416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906466f8-f92a-45fb-8146-bbeaad7f5527 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.391374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.392114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.392589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.392797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] PATH_INFO: `/v3/projects/2b37637885714d28bb2b6a2b4132de81/users/e84026fa4b80441c8d572bdf60eb6d7a/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.407805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b37637885714d28bb2b6a2b4132de81, user_id=e84026fa4b80441c8d572bdf60eb6d7a, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.411298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.424848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1229/2437] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:02 2022] PUT /identity/v3/projects/2b37637885714d28bb2b6a2b4132de81/users/e84026fa4b80441c8d572bdf60eb6d7a/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:02.488135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.488421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.489397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.516460 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] MFA Rules not processed for user `e84026fa4b80441c8d572bdf60eb6d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:02.517664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:02.580197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1209/2438] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:02.585186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.647047 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a3978-f575-46d6-ae11-1cfb520b3a33 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.651994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.652600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.652883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.653157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.653797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.655747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.687418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1230/2439] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:02.688639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.701986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930553a-3068-4af9-a071-3e94f5be0f3e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.763569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:02.769116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.770431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.770886 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.771398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.772100 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.774267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.797007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1210/2440] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:02 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:38:02.800954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.811266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7ba70-e60b-47d6-84cd-b45a5740753b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.815979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.816574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.816938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.817183 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.817833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.819463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.830035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1231/2441] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:02.834872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.845602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c10b2-d44a-423b-bd44-7dd38bf9bba8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.850827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.851500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.851783 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.852059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] PATH_INFO: `/v3/projects/7dc9dcd6d7534800b3fa09b4c577cc03/users/ceb7330151e649a999a7fa1a97f2a5bf/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.867236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc9dcd6d7534800b3fa09b4c577cc03, user_id=ceb7330151e649a999a7fa1a97f2a5bf, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.869910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.883641 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1211/2442] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:02 2022] PUT /identity/v3/projects/7dc9dcd6d7534800b3fa09b4c577cc03/users/ceb7330151e649a999a7fa1a97f2a5bf/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:02.887543 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.898115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fb7f2-833a-40fd-a3da-9ac354d4d8e4 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.955229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd971c5-80ff-4525-8784-627f435ed2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.955860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd971c5-80ff-4525-8784-627f435ed2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.956144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd971c5-80ff-4525-8784-627f435ed2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.956418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd971c5-80ff-4525-8784-627f435ed2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.957102 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd971c5-80ff-4525-8784-627f435ed2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:02.959438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd971c5-80ff-4525-8784-627f435ed2d9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:02.971559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1232/2443] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:02.977400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:02.989362 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31c75e-7fd1-4f3f-9aef-b7eebeeacf1e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:02.994203 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:02.994929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:02.995308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:02.995653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] PATH_INFO: `/v3/projects/7dc9dcd6d7534800b3fa09b4c577cc03/users/ceb7330151e649a999a7fa1a97f2a5bf/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:02.997563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc9dcd6d7534800b3fa09b4c577cc03, user_id=ceb7330151e649a999a7fa1a97f2a5bf, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:03.000225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:03.011730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1212/2444] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:02 2022] PUT /identity/v3/projects/7dc9dcd6d7534800b3fa09b4c577cc03/users/ceb7330151e649a999a7fa1a97f2a5bf/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:03.022762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:03.023075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:03.023398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:03.043754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:03.053737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] MFA Rules not processed for user `ceb7330151e649a999a7fa1a97f2a5bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:03.055331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:03.122783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1233/2445] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:03.133855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:03.135589 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:03.191177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98335f1d-f235-40da-a89d-d2c24905c512 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:03.195299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d2100-fe63-4663-8326-6ee5b9b6a9c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:03.195894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d2100-fe63-4663-8326-6ee5b9b6a9c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:03.196173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d2100-fe63-4663-8326-6ee5b9b6a9c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:03.196445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d2100-fe63-4663-8326-6ee5b9b6a9c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:03.205172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d2100-fe63-4663-8326-6ee5b9b6a9c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:03.206606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d2100-fe63-4663-8326-6ee5b9b6a9c4 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:03.216479 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1213/2446] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:03.348086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:03.410905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9ed00-d898-40f1-895a-09fc23b7a1c1 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:03.415619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:03.416245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:03.416529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:03.416806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] PATH_INFO: `/v3/users/e84026fa4b80441c8d572bdf60eb6d7a` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:03.417919 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e84026fa4b80441c8d572bdf60eb6d7a)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:03.419890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:03.469843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e84026fa4b80441c8d572bdf60eb6d7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:03.478708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e84026fa4b80441c8d572bdf60eb6d7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:03.480362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e84026fa4b80441c8d572bdf60eb6d7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:03.489893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e84026fa4b80441c8d572bdf60eb6d7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:03.512036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e84026fa4b80441c8d572bdf60eb6d7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:03.519880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1234/2447] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:03 2022] DELETE /identity/v3/users/e84026fa4b80441c8d572bdf60eb6d7a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:03.524491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:03.524787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:03.525067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:03.544253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:03.545444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:03.608372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1214/2448] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:38:03.618581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:03.620065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:03.634697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:03.679298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf7fe8-c55e-4d39-b342-03bbf83d8bdb admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:03.682852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252387cc-f258-4e3d-bd20-7121cdf6ab90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:03.683491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252387cc-f258-4e3d-bd20-7121cdf6ab90 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:03.683798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252387cc-f258-4e3d-bd20-7121cdf6ab90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:03.684069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252387cc-f258-4e3d-bd20-7121cdf6ab90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:03.692849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252387cc-f258-4e3d-bd20-7121cdf6ab90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:03.694258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252387cc-f258-4e3d-bd20-7121cdf6ab90 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:03.703546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1235/2449] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:04.190976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:04.193664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:04.211182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2352e-6926-4106-901b-76e98296feff None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:04.211698 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2352e-6926-4106-901b-76e98296feff None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:04.211805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2352e-6926-4106-901b-76e98296feff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:04.227127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e0057-5aca-4f7c-a745-0acdceee00ee None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:04.233109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-73d2352e-6926-4106-901b-76e98296feff None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:04.234323 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-73d2352e-6926-4106-901b-76e98296feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:04.244744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:04.245397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:04.245679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:04.245953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] PATH_INFO: `/v3/projects/2b37637885714d28bb2b6a2b4132de81` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:04.250963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b37637885714d28bb2b6a2b4132de81)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:04.253063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:04.298271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1236/2450] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 14:38:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:38:04.313060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2352e-6926-4106-901b-76e98296feff None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:04.314817 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-73d2352e-6926-4106-901b-76e98296feff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:04.335728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:04.349299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b37637885714d28bb2b6a2b4132de81', 'request_id': 'req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.357831 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b37637885714d28bb2b6a2b4132de81', 'request_id': 'req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.364968 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b37637885714d28bb2b6a2b4132de81', 'request_id': 'req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.386218 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] The token cache is being invalidate because project 2b37637885714d28bb2b6a2b4132de81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:04.387219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.389398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.396557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1215/2451] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:04 2022] DELETE /identity/v3/projects/2b37637885714d28bb2b6a2b4132de81 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:04.397599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:04.399186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:04.463416 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2352e-6926-4106-901b-76e98296feff None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:04.477963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a3c7-8ebd-463e-ba70-7ee6c12d7e04 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:04.483468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:04.484245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:04.484639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:04.484990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] PATH_INFO: `/v3/users/ceb7330151e649a999a7fa1a97f2a5bf` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:04.486913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb7330151e649a999a7fa1a97f2a5bf)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:04.489471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:04.539932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:04.540868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:04.541577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:04.542037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:04.544148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:04.553165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:04.560860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb7330151e649a999a7fa1a97f2a5bf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.569992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb7330151e649a999a7fa1a97f2a5bf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.578458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb7330151e649a999a7fa1a97f2a5bf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.579390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb7330151e649a999a7fa1a97f2a5bf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.603361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb7330151e649a999a7fa1a97f2a5bf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:04.617433 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1216/2452] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:04 2022] DELETE /identity/v3/users/ceb7330151e649a999a7fa1a97f2a5bf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:04.625428 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:04.627521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f35e7b-6429-4fbe-bf2c-9d13aa18755b service swift] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:04.638622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1237/2453] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:38:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 14:38:05.331644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.333263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:05.388735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:05.406394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44036a57-7e2d-4b92-9ea9-6a0de4ad18cf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.411732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.412357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.412635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.412909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] PATH_INFO: `/v3/projects/7dc9dcd6d7534800b3fa09b4c577cc03` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.414052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc9dcd6d7534800b3fa09b4c577cc03)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.415992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.460551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc9dcd6d7534800b3fa09b4c577cc03', 'request_id': 'req-6896ba9f-1c68-436e-8996-a5457d383f4b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:05.465960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc9dcd6d7534800b3fa09b4c577cc03', 'request_id': 'req-6896ba9f-1c68-436e-8996-a5457d383f4b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:05.471947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc9dcd6d7534800b3fa09b4c577cc03', 'request_id': 'req-6896ba9f-1c68-436e-8996-a5457d383f4b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:05.484546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] The token cache is being invalidate because project 7dc9dcd6d7534800b3fa09b4c577cc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:05.484884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:05.485518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6896ba9f-1c68-436e-8996-a5457d383f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:05.487015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1217/2454] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:05 2022] DELETE /identity/v3/projects/7dc9dcd6d7534800b3fa09b4c577cc03 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:05.492588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.493049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.493162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.512288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:05.513477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:05.570813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1238/2455] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:05.579967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.580310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.580617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.601587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:05.602781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:05.610445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1218/2456] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:05.614863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.624654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bafbe-1d46-48e4-a55d-f51e95d5cc88 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.629160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.629753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.630034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.630303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.630953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:05.631297 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.632884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.644391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1239/2457] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:05.649193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.658239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c088eec-da73-4624-8693-ab8817a1d9d2 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.663685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.664269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.664546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.664815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.665435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.667359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.694073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1219/2458] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:05 2022] POST /identity/v3/projects => generated 359 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:38:05.698168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.708500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db76c-def4-4ec4-a400-bdacbd197553 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.764755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.765282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.765559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.765829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.766460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.768405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.790639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1240/2459] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:05 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:05.794703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.805354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54f15a-0c4d-44ed-8870-8eba69bea8de admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.810133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.810827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.811230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.811572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.812317 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.814041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.827793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1220/2460] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:05.831589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.844407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3fcba-c325-4a8b-bbee-392aefe00970 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.850029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d727d4-7f25-44b9-8e02-ed4bd366366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.850712 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d727d4-7f25-44b9-8e02-ed4bd366366c admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.851060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d727d4-7f25-44b9-8e02-ed4bd366366c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.851443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d727d4-7f25-44b9-8e02-ed4bd366366c admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.852750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d727d4-7f25-44b9-8e02-ed4bd366366c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.854871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d727d4-7f25-44b9-8e02-ed4bd366366c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.876406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1241/2461] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:38:05.877639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:05.894073 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b10f46-0f5e-4347-bbea-1d915fd6b3ee admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:05.901290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:05.901983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.902259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.902532 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] PATH_INFO: `/v3/projects/4f1cd2e2e03e42c3afb840f02e56621c/users/bcae4074523b4ec1a1e97fc7f187c735/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.918034 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1cd2e2e03e42c3afb840f02e56621c, user_id=bcae4074523b4ec1a1e97fc7f187c735, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:05.920732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:05.934156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1221/2462] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:05 2022] PUT /identity/v3/projects/4f1cd2e2e03e42c3afb840f02e56621c/users/bcae4074523b4ec1a1e97fc7f187c735/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:05.944833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:05.945120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:05.945394 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:05.965634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:05.972987 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] MFA Rules not processed for user `bcae4074523b4ec1a1e97fc7f187c735`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:05.974751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:06.037641 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1242/2463] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:06.069606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:06.071357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:06.126000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34efe0-abc7-4d4b-ab5e-2a7c4e3fbf95 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:06.130054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa2947-eb13-4461-b62e-0397a8e73ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:06.130643 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2947-eb13-4461-b62e-0397a8e73ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:06.130923 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2947-eb13-4461-b62e-0397a8e73ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:06.131224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2947-eb13-4461-b62e-0397a8e73ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:06.139255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa2947-eb13-4461-b62e-0397a8e73ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:06.140673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa2947-eb13-4461-b62e-0397a8e73ee0 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:06.149860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1222/2464] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:06.166364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:06.226924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701b5ce-cbcf-4302-b9dc-e4ff8345d027 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:06.231562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:06.232177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:06.232459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:06.232732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] PATH_INFO: `/v3/users/bcae4074523b4ec1a1e97fc7f187c735` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:06.233836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcae4074523b4ec1a1e97fc7f187c735)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:06.235833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:06.280193 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcae4074523b4ec1a1e97fc7f187c735'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.287487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcae4074523b4ec1a1e97fc7f187c735'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.288275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcae4074523b4ec1a1e97fc7f187c735'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.295541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcae4074523b4ec1a1e97fc7f187c735'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.308380 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcae4074523b4ec1a1e97fc7f187c735'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.314174 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1243/2465] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:06 2022] DELETE /identity/v3/users/bcae4074523b4ec1a1e97fc7f187c735 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:06.319132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:06.319422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:06.319695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:06.338663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:06.339910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:06.351371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:06.427341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1223/2466] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:06.443111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:06.444592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:06.467184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:06.508520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb4f36-f32c-4f46-b5bd-bc8ebd6b531c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:06.512242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7f186-e238-464a-ae18-464918b3ff39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:06.512855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7f186-e238-464a-ae18-464918b3ff39 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:06.513131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7f186-e238-464a-ae18-464918b3ff39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:06.513399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7f186-e238-464a-ae18-464918b3ff39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:06.522695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7f186-e238-464a-ae18-464918b3ff39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:06.524159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7f186-e238-464a-ae18-464918b3ff39 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:06.533664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1244/2467] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:06.808535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:06.809952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:06.823342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb5072-8b8e-4022-96e1-64f927cc64aa None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:06.828829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:06.829789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:06.830237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:06.830665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] PATH_INFO: `/v3/projects/4f1cd2e2e03e42c3afb840f02e56621c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:06.831981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1cd2e2e03e42c3afb840f02e56621c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:06.835552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:06.948411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1cd2e2e03e42c3afb840f02e56621c', 'request_id': 'req-335fd85f-5453-4863-b343-3b798fb22266'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.954342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1cd2e2e03e42c3afb840f02e56621c', 'request_id': 'req-335fd85f-5453-4863-b343-3b798fb22266'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.960855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1cd2e2e03e42c3afb840f02e56621c', 'request_id': 'req-335fd85f-5453-4863-b343-3b798fb22266'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.982118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] The token cache is being invalidate because project 4f1cd2e2e03e42c3afb840f02e56621c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:06.982422 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.985062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:06.986717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1224/2468] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:06 2022] DELETE /identity/v3/projects/4f1cd2e2e03e42c3afb840f02e56621c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:08.304662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:08.305465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:08.305741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:08.325026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] MFA Rules not processed for user `b644a4d4db88473989cd032bbc16b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:08.326204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:08.387798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1245/2469] 10.222.0.50 () {58 vars in 967 bytes} [Mon Aug 8 14:38:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:08.402623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:08.404762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:08.490232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fd85f-5453-4863-b343-3b798fb22266 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:08.499065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94cf8e7d52a3441196ac4b4a63fb13ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:08.499760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:08.500053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:08.500396 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:08.509788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:08.511623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:08.521233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1225/2470] 10.222.0.50 () {60 vars in 1384 bytes} [Mon Aug 8 14:38:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 14:38:12.052856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:12.054402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:12.129485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9264-ff39-4ee9-9a14-3a0e8618106b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:12.134488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:12.135121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:12.135398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:12.135670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:12.137377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:12.149251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:12.150814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:12.161421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1246/2471] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:14.056706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:14.058170 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:14.096557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:14.124944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc187d-73d0-4ffc-8769-fa089e8648eb service swift] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:14.128685 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:14.129932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:14.130215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:14.130487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:14.132272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:14.195184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:14.196575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:14.206335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1226/2472] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:14.473872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:14.477673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:14.555833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7daeb2-54ee-4e8e-9d9b-f401da68a5af service placement] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:14.561453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:14.562210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:14.562492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:14.562769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:14.564630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:14.576485 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:14.637026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:14.639556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:14.652985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1247/2473] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:15.172378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:15.174018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:15.193384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef5c0c-f2fb-4aaf-a989-e9ae38ac7d21 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:15.200224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:15.200871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:15.201149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:15.201426 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] PATH_INFO: `/v3/users/3cc610fe5d814027b01236949e145430` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:15.202577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc610fe5d814027b01236949e145430)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:15.204770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:15.277636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc610fe5d814027b01236949e145430'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.286290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc610fe5d814027b01236949e145430'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.301792 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc610fe5d814027b01236949e145430'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.302915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc610fe5d814027b01236949e145430'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.317481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc610fe5d814027b01236949e145430'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.323875 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1227/2474] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:15 2022] DELETE /identity/v3/users/3cc610fe5d814027b01236949e145430 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:15.755153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:15.768402 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28199c8-1c19-4dd5-b16d-edbd510cc4be service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:15.835057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:15.836426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:15.836956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:15.837456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] PATH_INFO: `/v3/projects/90d6ef85228f4676b9ac19e123e1462a` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:15.840188 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90d6ef85228f4676b9ac19e123e1462a)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:15.842570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:15.900611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d6ef85228f4676b9ac19e123e1462a', 'request_id': 'req-6dca0255-aaa0-4c47-a435-9b33119e3e08'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.906975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d6ef85228f4676b9ac19e123e1462a', 'request_id': 'req-6dca0255-aaa0-4c47-a435-9b33119e3e08'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.915129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d6ef85228f4676b9ac19e123e1462a', 'request_id': 'req-6dca0255-aaa0-4c47-a435-9b33119e3e08'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.930109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] The token cache is being invalidate because project 90d6ef85228f4676b9ac19e123e1462a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:15.930397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.931176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:15.932749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1248/2475] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:15 2022] DELETE /identity/v3/projects/90d6ef85228f4676b9ac19e123e1462a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:15.937974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:15.938536 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:15.957759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962639b-9feb-40c7-a836-cca405cded3d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:16.018193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:16.033818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:16.034275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:16.034650 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:16.035551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] PATH_INFO: `/v3/users/42859720d5fe4a609a4eca452a80bef2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:16.037707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42859720d5fe4a609a4eca452a80bef2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:16.039933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:16.094080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42859720d5fe4a609a4eca452a80bef2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.102553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42859720d5fe4a609a4eca452a80bef2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.111602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42859720d5fe4a609a4eca452a80bef2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.112456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42859720d5fe4a609a4eca452a80bef2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.126939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42859720d5fe4a609a4eca452a80bef2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.137430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1228/2476] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:15 2022] DELETE /identity/v3/users/42859720d5fe4a609a4eca452a80bef2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:16.326948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:16.328516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:16.358535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca0255-aaa0-4c47-a435-9b33119e3e08 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:16.411163 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:16.438852 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:16.439538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:16.439819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:16.440089 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] PATH_INFO: `/v3/projects/37963517458a4a8e8291a173befcdeaf` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:16.441254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37963517458a4a8e8291a173befcdeaf)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:16.443202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:16.493777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37963517458a4a8e8291a173befcdeaf', 'request_id': 'req-cb188886-e396-49a2-a5cd-281953195bb7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.499549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37963517458a4a8e8291a173befcdeaf', 'request_id': 'req-cb188886-e396-49a2-a5cd-281953195bb7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.505853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37963517458a4a8e8291a173befcdeaf', 'request_id': 'req-cb188886-e396-49a2-a5cd-281953195bb7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.517889 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] The token cache is being invalidate because project 37963517458a4a8e8291a173befcdeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:16.518182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.518810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:16.520388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1249/2477] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:16 2022] DELETE /identity/v3/projects/37963517458a4a8e8291a173befcdeaf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:17.653376 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:17.654882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:17.668418 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641e4c-abb7-445a-a38a-30e0b428cb98 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:17.724821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:17.725455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:17.725737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:17.726014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] PATH_INFO: `/v3/users/75b75be070a147fba4788b0f51d8d0fb` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:17.727257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75b75be070a147fba4788b0f51d8d0fb)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:17.729192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:17.773233 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75b75be070a147fba4788b0f51d8d0fb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:17.779966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75b75be070a147fba4788b0f51d8d0fb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:17.787137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75b75be070a147fba4788b0f51d8d0fb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:17.787803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75b75be070a147fba4788b0f51d8d0fb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:17.794727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:17.801094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d7dfef9b-7344-4d75-8bf8-b1c3e3ceb483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75b75be070a147fba4788b0f51d8d0fb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:17.808329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1229/2478] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:17 2022] DELETE /identity/v3/users/75b75be070a147fba4788b0f51d8d0fb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:17.979766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:17.981843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:17.995782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb188886-e396-49a2-a5cd-281953195bb7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.060189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.060762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.061132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.061411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] PATH_INFO: `/v3/projects/0400861a88074a1480b54ce5e651cc76` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.062598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0400861a88074a1480b54ce5e651cc76)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.064552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.109436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0400861a88074a1480b54ce5e651cc76', 'request_id': 'req-4936410e-c619-47db-8035-799037cd02d3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:18.115300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0400861a88074a1480b54ce5e651cc76', 'request_id': 'req-4936410e-c619-47db-8035-799037cd02d3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:18.121285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0400861a88074a1480b54ce5e651cc76', 'request_id': 'req-4936410e-c619-47db-8035-799037cd02d3'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:18.132767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] The token cache is being invalidate because project 0400861a88074a1480b54ce5e651cc76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:18.133062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:18.133680 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-4936410e-c619-47db-8035-799037cd02d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:18.135177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1250/2479] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:17 2022] DELETE /identity/v3/projects/0400861a88074a1480b54ce5e651cc76 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:18.140005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.140291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.140563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.159503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:18.161449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:18.218720 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1230/2480] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:18.226990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.227314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.227578 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.235197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:18.249390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:18.250654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:18.258648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1251/2481] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:18.262715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.272922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fc7f1-2c76-4cd2-898a-bed8dfb33aed None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.277468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.278057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.278335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.278604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.279274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:18.279577 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.281160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.293276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1231/2482] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:18.297469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.306711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e46967-9e11-4831-9d6b-1a867817f9d8 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.311379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.311918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.312190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.312465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.313084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.314979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.341786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1252/2483] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:18.345765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.356094 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e911ca-e000-4b79-b391-19e7d71297bc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.411439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.412095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.412452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.412719 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.413332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.415295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.435360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1232/2484] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:18.439406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.449639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbe38a-0004-4d9c-a21f-753232973a2a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.454253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.454841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.455147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.455422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.456050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.458129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.467907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1253/2485] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:38:18.471481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.481844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53774a-85c1-4718-bcfe-a5e5165873ff admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.486380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122492ff-d17b-4c48-a8de-d473bf32e22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.486958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122492ff-d17b-4c48-a8de-d473bf32e22f admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.487268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122492ff-d17b-4c48-a8de-d473bf32e22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.487541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122492ff-d17b-4c48-a8de-d473bf32e22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.488172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122492ff-d17b-4c48-a8de-d473bf32e22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.489739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122492ff-d17b-4c48-a8de-d473bf32e22f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.500437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1233/2486] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:18.504125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.513989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ad03a-9531-41be-b2c7-c7f7a22e0f29 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.518657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.519808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.520084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.520352 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] PATH_INFO: `/v3/projects/0ccc80a2ec894bbab5202623ca88f587/users/e2afa0dadde448178e825f3a7757b7d6/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.534436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccc80a2ec894bbab5202623ca88f587, user_id=e2afa0dadde448178e825f3a7757b7d6, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.537870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.550270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1254/2487] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:18 2022] PUT /identity/v3/projects/0ccc80a2ec894bbab5202623ca88f587/users/e2afa0dadde448178e825f3a7757b7d6/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:18.561824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.562050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.562329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.591370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] MFA Rules not processed for user `e2afa0dadde448178e825f3a7757b7d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:18.592588 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:18.638063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:18.661713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1234/2488] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:18.666296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.730856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0c431-1813-4c15-8a9d-9aa631c31947 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.735871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.736480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.736759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.737156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.737777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.740598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.767652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1255/2489] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:18.772356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.784847 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90a892-28e0-4e18-97d0-282252af08b3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.841616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.842219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.842497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.842766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.843432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.845331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.867762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1235/2490] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:18 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:18.871683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.882510 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40119283-f2bc-438c-977b-66faea0bfc31 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.887042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.887651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.887927 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.888195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.888833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.890460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.901296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1256/2491] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:18.905585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.915983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0ed67-a0e4-40e2-ab24-e5980c10ef1d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:18.920570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:18.921224 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:18.921493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:18.921771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] PATH_INFO: `/v3/projects/222831d9835c423fb57a8233dbe2971e/users/3d3a869e7d8048b2a8f92a52666f7afc/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:18.936208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222831d9835c423fb57a8233dbe2971e, user_id=3d3a869e7d8048b2a8f92a52666f7afc, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:18.938857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:18.951630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1236/2492] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:18 2022] PUT /identity/v3/projects/222831d9835c423fb57a8233dbe2971e/users/3d3a869e7d8048b2a8f92a52666f7afc/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:18.955530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:18.966017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd681c2-6f3f-4819-972b-2aca6e161342 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:19.021042 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc3884-a547-423f-a2a1-190b814c452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:19.021690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc3884-a547-423f-a2a1-190b814c452f admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:19.021914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc3884-a547-423f-a2a1-190b814c452f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:19.022192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc3884-a547-423f-a2a1-190b814c452f admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:19.022838 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc3884-a547-423f-a2a1-190b814c452f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:19.024462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc3884-a547-423f-a2a1-190b814c452f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:19.034280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1257/2493] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:19.038498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:19.049112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38298943-e6d2-4ee2-ad4c-bbbf518fae77 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:19.054033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:19.054674 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:19.054953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:19.055250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] PATH_INFO: `/v3/projects/222831d9835c423fb57a8233dbe2971e/users/3d3a869e7d8048b2a8f92a52666f7afc/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:19.057112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222831d9835c423fb57a8233dbe2971e, user_id=3d3a869e7d8048b2a8f92a52666f7afc, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:19.059874 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:19.070832 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1237/2494] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:19 2022] PUT /identity/v3/projects/222831d9835c423fb57a8233dbe2971e/users/3d3a869e7d8048b2a8f92a52666f7afc/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:19.080961 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:19.081247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:19.081522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:19.100942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:19.110868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] MFA Rules not processed for user `3d3a869e7d8048b2a8f92a52666f7afc`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:19.112793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:19.182994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1258/2495] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:19.194762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:19.196648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:19.257028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ff0d9-1c58-4a2f-ab4c-cc9301610bf1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:19.260909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294de6e4-1dce-4541-81dd-836b5dd173c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:19.261519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294de6e4-1dce-4541-81dd-836b5dd173c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:19.261797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294de6e4-1dce-4541-81dd-836b5dd173c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:19.262065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294de6e4-1dce-4541-81dd-836b5dd173c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:19.271263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294de6e4-1dce-4541-81dd-836b5dd173c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:19.272689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294de6e4-1dce-4541-81dd-836b5dd173c7 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:19.282360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1238/2496] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:19.332540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:19.341913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ea527-1fe1-44ea-ac6f-37f7099b4f59 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:19.345545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:19.346081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:19.346355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:19.346620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:19.413105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:19.414656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:19.425068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1259/2497] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:21.598903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb38f-ed41-46b1-96d5-77e5f8fd928f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:21.599449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb38f-ed41-46b1-96d5-77e5f8fd928f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:21.599547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcb38f-ed41-46b1-96d5-77e5f8fd928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:21.619048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-adbcb38f-ed41-46b1-96d5-77e5f8fd928f None None] MFA Rules not processed for user `2495d3c63dc44049872afb908ca6133b`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:21.620283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-adbcb38f-ed41-46b1-96d5-77e5f8fd928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:21.652997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:21.675887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1239/2498] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:22.733263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:22.805729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d3116-e599-48e5-959c-d6bd461a2bc5 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:22.810812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:22.811388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:22.811693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:22.811960 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] PATH_INFO: `/v3/users/e2afa0dadde448178e825f3a7757b7d6` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:22.813096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2afa0dadde448178e825f3a7757b7d6)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:22.815002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:22.855521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:22.867288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2afa0dadde448178e825f3a7757b7d6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:22.875587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2afa0dadde448178e825f3a7757b7d6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:22.876463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2afa0dadde448178e825f3a7757b7d6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:22.884808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2afa0dadde448178e825f3a7757b7d6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:22.897789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2afa0dadde448178e825f3a7757b7d6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:22.903937 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1260/2499] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:22 2022] DELETE /identity/v3/users/e2afa0dadde448178e825f3a7757b7d6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:22.908435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:22.908904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:22.908998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:22.930109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:22.931342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:22.991016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1240/2500] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:23.002093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:23.003849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:23.062227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7726395-c5e9-443a-8f92-727ee1403921 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:23.065768 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:23.066365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:23.066643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:23.066913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:23.075705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:23.077121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:23.086976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1261/2501] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:23.358884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:23.360377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:23.373704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d452b27-6cd2-48fa-9fc6-7ba42d35cf97 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:23.380465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:23.381261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:23.381656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:23.381991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] PATH_INFO: `/v3/projects/0ccc80a2ec894bbab5202623ca88f587` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:23.383420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccc80a2ec894bbab5202623ca88f587)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:23.385633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:23.436196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccc80a2ec894bbab5202623ca88f587', 'request_id': 'req-051a2e1b-c709-4f0d-bf55-e7d2a6606996'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.443138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccc80a2ec894bbab5202623ca88f587', 'request_id': 'req-051a2e1b-c709-4f0d-bf55-e7d2a6606996'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.449360 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccc80a2ec894bbab5202623ca88f587', 'request_id': 'req-051a2e1b-c709-4f0d-bf55-e7d2a6606996'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.464827 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] The token cache is being invalidate because project 0ccc80a2ec894bbab5202623ca88f587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:23.465183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.465739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.467393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1241/2502] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:23 2022] DELETE /identity/v3/projects/0ccc80a2ec894bbab5202623ca88f587 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:23.471374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:23.483864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788e702-89a8-4ff4-9166-199da30a8a60 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:23.551221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:23.552095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:23.552095 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:23.552369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] PATH_INFO: `/v3/users/3d3a869e7d8048b2a8f92a52666f7afc` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:23.555456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3a869e7d8048b2a8f92a52666f7afc)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:23.555456 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:23.625003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3a869e7d8048b2a8f92a52666f7afc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.634626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3a869e7d8048b2a8f92a52666f7afc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.635286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3a869e7d8048b2a8f92a52666f7afc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.638355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:23.645628 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3a869e7d8048b2a8f92a52666f7afc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.663986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7d640112-da75-4b2d-a2bd-fde3c032423a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3a869e7d8048b2a8f92a52666f7afc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:23.671064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1262/2503] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:23 2022] DELETE /identity/v3/users/3d3a869e7d8048b2a8f92a52666f7afc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:23.956143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:23.957760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:23.976226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a2e1b-c709-4f0d-bf55-e7d2a6606996 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.043298 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.043943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.044225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.044500 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] PATH_INFO: `/v3/projects/222831d9835c423fb57a8233dbe2971e` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.045717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222831d9835c423fb57a8233dbe2971e)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.048018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.063625 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:24.107517 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222831d9835c423fb57a8233dbe2971e', 'request_id': 'req-5160181e-84d3-43ee-9595-0150218938e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:24.117900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222831d9835c423fb57a8233dbe2971e', 'request_id': 'req-5160181e-84d3-43ee-9595-0150218938e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:24.126129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222831d9835c423fb57a8233dbe2971e', 'request_id': 'req-5160181e-84d3-43ee-9595-0150218938e7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:24.140126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] The token cache is being invalidate because project 222831d9835c423fb57a8233dbe2971e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:24.140425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:24.141038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5160181e-84d3-43ee-9595-0150218938e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:24.142738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1242/2504] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:23 2022] DELETE /identity/v3/projects/222831d9835c423fb57a8233dbe2971e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:24.147661 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.148191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.148191 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.169499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:24.170702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:24.262945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1263/2505] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:24.271010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.271338 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.271604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.292964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:24.294270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:24.303202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1243/2506] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:24.308031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.319282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212732b8-76f1-4382-ac3e-45775ae5a656 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.324203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.324791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.325072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.325343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.326008 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:24.326309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.327957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.342318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1264/2507] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:38:24.346762 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.357711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb116fa-5280-4eb9-9b7d-bf3cb6ccf6a6 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.363190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.363864 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.364147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.364424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.365075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.367118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.398208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1244/2508] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/projects => generated 409 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:24.402453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.413880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d1699-3377-4cdf-8a15-32aa8e4d7eda admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.496683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.497284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.497565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.497836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.499106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.500408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.525835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1265/2509] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/users => generated 305 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:24.527128 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.538860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d37d-d1d7-4cd7-80d2-4735fe47ba63 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.543580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.544205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.544548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.544818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.545456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.547047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.556965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1245/2510] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:24.560895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.570847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc175262-79bf-40e8-9b35-1419f036ddef admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.575461 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de093d-8128-44e5-b274-1a49fc49ae1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.576086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de093d-8128-44e5-b274-1a49fc49ae1a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.576364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de093d-8128-44e5-b274-1a49fc49ae1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.576635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de093d-8128-44e5-b274-1a49fc49ae1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.577268 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de093d-8128-44e5-b274-1a49fc49ae1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.578984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de093d-8128-44e5-b274-1a49fc49ae1a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.592935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1266/2511] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:24.596044 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.606934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c042a61-af13-4c8e-b1dd-0c2f0555d58e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.634071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.634870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.635387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.636220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] PATH_INFO: `/v3/projects/2bc9827b28214e5a9303a54d2e14fa5d/users/ea8d6b7f0bb04b94b7ea95b68e563c6c/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.655137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc9827b28214e5a9303a54d2e14fa5d, user_id=ea8d6b7f0bb04b94b7ea95b68e563c6c, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.657979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.674716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1246/2512] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:24 2022] PUT /identity/v3/projects/2bc9827b28214e5a9303a54d2e14fa5d/users/ea8d6b7f0bb04b94b7ea95b68e563c6c/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:24.686190 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.686482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.686760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.717387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] MFA Rules not processed for user `ea8d6b7f0bb04b94b7ea95b68e563c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:24.719350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:24.749689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:24.791723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1267/2513] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:24.796294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.862018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae3faa-1c6f-4479-b82e-ace90d7c1934 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.867434 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.868048 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.868411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.868706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.869239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.871223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:24.906279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1247/2514] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:38:24.910382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:24.921997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b567a-82fc-4e44-b34e-4f48363ae9e5 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:24.980798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:24.981412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:24.981693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:24.981964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:24.982590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:24.984580 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.008724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1268/2515] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:24 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:25.012779 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:25.025370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df21e-15db-4a53-a890-fdbd63230992 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:25.030505 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:25.031221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.031570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.031912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.032636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:25.034415 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.046919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1248/2516] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:25.051911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:25.064081 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609e724-c9d1-41f3-aaae-5f9b27261ecf admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:25.069481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:25.070136 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.070418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.070691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] PATH_INFO: `/v3/projects/41e93c458b09481b9af7faf2b705c389/users/98ec6dcfef0342ad93952e88c89bc0d7/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.087150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e93c458b09481b9af7faf2b705c389, user_id=98ec6dcfef0342ad93952e88c89bc0d7, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:25.089819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.103351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1269/2517] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:25 2022] PUT /identity/v3/projects/41e93c458b09481b9af7faf2b705c389/users/98ec6dcfef0342ad93952e88c89bc0d7/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:25.107307 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:25.118127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a606d60-c21f-4f3c-a7e6-1407dad78cb8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:25.123075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:25.179698 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7c537-f0f9-423b-a867-98a5ffd819cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:25.180303 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c537-f0f9-423b-a867-98a5ffd819cf admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.180584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c537-f0f9-423b-a867-98a5ffd819cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.180857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7c537-f0f9-423b-a867-98a5ffd819cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.181493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7c537-f0f9-423b-a867-98a5ffd819cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:25.183111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7c537-f0f9-423b-a867-98a5ffd819cf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.193655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1249/2518] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:25.197907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:25.208087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc795-2786-4920-8079-6c2b51e3cf28 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:25.212614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:25.213257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.213531 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.213801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] PATH_INFO: `/v3/projects/41e93c458b09481b9af7faf2b705c389/users/98ec6dcfef0342ad93952e88c89bc0d7/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.215737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e93c458b09481b9af7faf2b705c389, user_id=98ec6dcfef0342ad93952e88c89bc0d7, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:25.218347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.228658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1270/2519] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:25 2022] PUT /identity/v3/projects/41e93c458b09481b9af7faf2b705c389/users/98ec6dcfef0342ad93952e88c89bc0d7/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:25.239297 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.239586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.239943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.267764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] MFA Rules not processed for user `98ec6dcfef0342ad93952e88c89bc0d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:25.269038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:25.337152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1250/2520] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:25.347812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:25.349573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:25.407011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d59c1-849b-468f-8fa2-0acc3ccdc314 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:25.411429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:25.412026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.412305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.412568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.421535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:25.422949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.434165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1271/2521] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:25.582498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:25.593327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd77d7-a2e3-4247-ab7b-f1319a7798cc None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:25.597541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37b78f-d9fa-4904-86d8-45c40de8b586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:25.598142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b78f-d9fa-4904-86d8-45c40de8b586 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:25.598419 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b78f-d9fa-4904-86d8-45c40de8b586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:25.598689 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37b78f-d9fa-4904-86d8-45c40de8b586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:25.652756 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b78f-d9fa-4904-86d8-45c40de8b586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:25.654174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37b78f-d9fa-4904-86d8-45c40de8b586 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:25.663612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1251/2522] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:25 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:34.018512 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:34.045776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:34.080786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7cd87-3373-4acf-9019-e4b6d083f74f service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:34.085820 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:34.086433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:34.086736 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:34.087005 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] PATH_INFO: `/v3/users/ea8d6b7f0bb04b94b7ea95b68e563c6c` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:34.088150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8d6b7f0bb04b94b7ea95b68e563c6c)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:34.090074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:34.134445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8d6b7f0bb04b94b7ea95b68e563c6c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.141087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8d6b7f0bb04b94b7ea95b68e563c6c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.141758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8d6b7f0bb04b94b7ea95b68e563c6c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.149344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8d6b7f0bb04b94b7ea95b68e563c6c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.161914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8d6b7f0bb04b94b7ea95b68e563c6c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.167597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1272/2523] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:34 2022] DELETE /identity/v3/users/ea8d6b7f0bb04b94b7ea95b68e563c6c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:34.172017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:34.172512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:34.172609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:34.191051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:34.192250 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:34.220449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:34.260354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1252/2524] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:34.271155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:34.272599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:34.336914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1eb107-6f31-4508-933d-8ae1d5ec5edd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:34.341109 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:34.341727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:34.342006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:34.342276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:34.353819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:34.355421 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:34.367061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1273/2525] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:34.647876 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:34.649645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:34.665043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174c4bf-1765-4835-ad33-3d14e9f9827d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:34.670411 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:34.671066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:34.671404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:34.671675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] PATH_INFO: `/v3/projects/2bc9827b28214e5a9303a54d2e14fa5d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:34.672797 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc9827b28214e5a9303a54d2e14fa5d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:34.674693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:34.729327 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc9827b28214e5a9303a54d2e14fa5d', 'request_id': 'req-3454b559-3768-4eee-8f7a-bd0b8e7ae165'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.734661 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc9827b28214e5a9303a54d2e14fa5d', 'request_id': 'req-3454b559-3768-4eee-8f7a-bd0b8e7ae165'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.741398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc9827b28214e5a9303a54d2e14fa5d', 'request_id': 'req-3454b559-3768-4eee-8f7a-bd0b8e7ae165'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.757155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] The token cache is being invalidate because project 2bc9827b28214e5a9303a54d2e14fa5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:34.757477 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.758084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.759594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1253/2526] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:34 2022] DELETE /identity/v3/projects/2bc9827b28214e5a9303a54d2e14fa5d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:34.763659 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:34.773569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b725eaf-184c-4374-8f65-030f853ed45d service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:34.816542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:34.830984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:34.832202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:34.832482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:34.832754 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] PATH_INFO: `/v3/users/98ec6dcfef0342ad93952e88c89bc0d7` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:34.833893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ec6dcfef0342ad93952e88c89bc0d7)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:34.835833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:34.885526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ec6dcfef0342ad93952e88c89bc0d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.893013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ec6dcfef0342ad93952e88c89bc0d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.894226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ec6dcfef0342ad93952e88c89bc0d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.901138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ec6dcfef0342ad93952e88c89bc0d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.913920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b41569bb-2f7e-40e8-9fe0-c26c380f6b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ec6dcfef0342ad93952e88c89bc0d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:34.920762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1274/2527] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:34 2022] DELETE /identity/v3/users/98ec6dcfef0342ad93952e88c89bc0d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:35.211960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.214042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:35.228166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454b559-3768-4eee-8f7a-bd0b8e7ae165 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.292718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.294137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.294464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.294739 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] PATH_INFO: `/v3/projects/41e93c458b09481b9af7faf2b705c389` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.295970 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e93c458b09481b9af7faf2b705c389)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.297896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.346036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e93c458b09481b9af7faf2b705c389', 'request_id': 'req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:35.350104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:35.352359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e93c458b09481b9af7faf2b705c389', 'request_id': 'req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:35.361268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e93c458b09481b9af7faf2b705c389', 'request_id': 'req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:35.375148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] The token cache is being invalidate because project 41e93c458b09481b9af7faf2b705c389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:35.375447 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:35.376075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-75400c9b-e6b7-4a9c-9139-86ac8ce183a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:35.377559 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1254/2528] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:35 2022] DELETE /identity/v3/projects/41e93c458b09481b9af7faf2b705c389 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:35.382872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.383366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.383502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.403828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:35.405030 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:35.466703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1275/2529] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:38:35.475761 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.476051 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.476329 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.497535 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:35.498759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:35.507595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1255/2530] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:35.511879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.521405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910d78a-ab4e-4325-90c0-78ac7cbed825 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.526104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.526703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.526986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.527290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.527945 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:35.528251 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.529849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.541777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1276/2531] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:35.545918 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.556008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da04c87-3a62-400c-aac3-0ff82dc61119 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.560734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.561332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.561611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.561882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.562640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.564591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.592637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1256/2532] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:35 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:35.596731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.606842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa194b0-afaa-4390-84bb-4e7d81d36dcc admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.656417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:35.667514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.668126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.668439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.668717 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.669408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.671398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.693453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1277/2533] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:35 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:35.697765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.708654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517c08b-384b-400b-925b-116beee010cf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.713235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.713823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.714099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.714368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.715006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.716618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.726493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1257/2534] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:35.730342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.740869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a722a2b-8d10-46ca-b77c-82dc8f003636 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.745536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b9ab2-574e-4522-926e-1a1bdb02ab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.746127 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b9ab2-574e-4522-926e-1a1bdb02ab23 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.746412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b9ab2-574e-4522-926e-1a1bdb02ab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.746689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b9ab2-574e-4522-926e-1a1bdb02ab23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.747366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b9ab2-574e-4522-926e-1a1bdb02ab23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.748951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b9ab2-574e-4522-926e-1a1bdb02ab23 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.760053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1278/2535] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:38:35.763917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.773732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d372-dcc0-4b8a-8ad7-f0f9b66df793 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.778339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.778982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.779300 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.779575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] PATH_INFO: `/v3/projects/7e11442ca5d44c3398b028c52b30b6cf/users/ab897a78bb0e4df7b30b97ba0ebcac81/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.793991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e11442ca5d44c3398b028c52b30b6cf, user_id=ab897a78bb0e4df7b30b97ba0ebcac81, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.796713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:35.809945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1258/2536] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:35 2022] PUT /identity/v3/projects/7e11442ca5d44c3398b028c52b30b6cf/users/ab897a78bb0e4df7b30b97ba0ebcac81/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:35.820769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.821192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.821474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.849399 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] MFA Rules not processed for user `ab897a78bb0e4df7b30b97ba0ebcac81`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:35.850603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:35.913534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1279/2537] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:35.917990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:35.979803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735714a2-d856-4704-97ee-a95ca5a434c5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:35.984429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:35.985030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:35.985315 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:35.985593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:35.986208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:35.988746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.013969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1259/2538] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:35 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:36.017948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.028648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba5ed1-9ac4-47d3-a586-60f2e86cbc3a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:36.084244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:36.084841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.085119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.085387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.085994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:36.087915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.108705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1280/2539] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:36 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:36.113508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.124261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ef858-0a98-43db-8f3a-65f12047b1be admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:36.129160 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:36.129782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.130058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.130325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.130991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:36.132601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.142833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1260/2540] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:36.146847 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.156643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b2b10-5642-4a15-ac3b-afa809948d6d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:36.162013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:36.162648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.162930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.163241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] PATH_INFO: `/v3/projects/7e6119cfb913409dad86a87cbec8567e/users/d5885b770193431da1a0e1e33857bde1/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.178057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6119cfb913409dad86a87cbec8567e, user_id=d5885b770193431da1a0e1e33857bde1, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:36.180735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.193503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1281/2541] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:36 2022] PUT /identity/v3/projects/7e6119cfb913409dad86a87cbec8567e/users/d5885b770193431da1a0e1e33857bde1/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:36.197343 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.207646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d644b-25d0-4f06-93b3-3a81998e3aed admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:36.256841 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:36.267499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047687c0-4bb9-466a-9bd0-1753a45ba0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:36.268125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047687c0-4bb9-466a-9bd0-1753a45ba0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.268408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047687c0-4bb9-466a-9bd0-1753a45ba0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.269214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047687c0-4bb9-466a-9bd0-1753a45ba0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.269879 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047687c0-4bb9-466a-9bd0-1753a45ba0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:36.271491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047687c0-4bb9-466a-9bd0-1753a45ba0d2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.281147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1261/2542] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:36.286060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.296907 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a3cbc-90a7-4aa1-8d0c-0a67ab2a435a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:36.301530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:36.302158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.302442 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.302714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] PATH_INFO: `/v3/projects/7e6119cfb913409dad86a87cbec8567e/users/d5885b770193431da1a0e1e33857bde1/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.304645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6119cfb913409dad86a87cbec8567e, user_id=d5885b770193431da1a0e1e33857bde1, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:36.307288 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.317224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1282/2543] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:36 2022] PUT /identity/v3/projects/7e6119cfb913409dad86a87cbec8567e/users/d5885b770193431da1a0e1e33857bde1/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:36.327251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.327540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.327812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.355665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] MFA Rules not processed for user `d5885b770193431da1a0e1e33857bde1`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:36.357102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:36.426172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1262/2544] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:36.437249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.439158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:36.453587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:36.497150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548a087-c834-462a-88a1-92e6e2977991 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:36.500898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d77dd2-d7b7-44d3-b6e7-a33042051c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:36.501526 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77dd2-d7b7-44d3-b6e7-a33042051c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:36.501802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77dd2-d7b7-44d3-b6e7-a33042051c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:36.502073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77dd2-d7b7-44d3-b6e7-a33042051c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:36.510897 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d77dd2-d7b7-44d3-b6e7-a33042051c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:36.512305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d77dd2-d7b7-44d3-b6e7-a33042051c74 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:36.522023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1283/2545] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:36.935994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:36.997114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae6d0e-f78d-4b6f-8c7a-1955bb992c2f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:37.001607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:37.002225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:37.002504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:37.002775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] PATH_INFO: `/v3/users/ab897a78bb0e4df7b30b97ba0ebcac81` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:37.004557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab897a78bb0e4df7b30b97ba0ebcac81)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:37.006462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:37.052778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab897a78bb0e4df7b30b97ba0ebcac81'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.059603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab897a78bb0e4df7b30b97ba0ebcac81'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.067439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab897a78bb0e4df7b30b97ba0ebcac81'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.068052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab897a78bb0e4df7b30b97ba0ebcac81'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.077868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:37.080205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab897a78bb0e4df7b30b97ba0ebcac81'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.087962 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1263/2546] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:36 2022] DELETE /identity/v3/users/ab897a78bb0e4df7b30b97ba0ebcac81 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:37.092443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:37.092730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:37.093032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:37.111242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:37.112431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:37.169462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1284/2547] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:37.179693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:37.181091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:37.237220 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061bb3-e665-4f57-8cae-f82bcfe161b8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:37.241320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:37.241920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:37.242204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:37.242476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:37.251576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:37.253005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:37.262897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1264/2548] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:37.541957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:37.543423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:37.556338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52f88e-7996-4986-8e57-b588f8824805 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:37.561017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:37.561649 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:37.561931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:37.562226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] PATH_INFO: `/v3/projects/7e11442ca5d44c3398b028c52b30b6cf` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:37.563520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e11442ca5d44c3398b028c52b30b6cf)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:37.565431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:37.612739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e11442ca5d44c3398b028c52b30b6cf', 'request_id': 'req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.618930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e11442ca5d44c3398b028c52b30b6cf', 'request_id': 'req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.625527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e11442ca5d44c3398b028c52b30b6cf', 'request_id': 'req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.637449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] The token cache is being invalidate because project 7e11442ca5d44c3398b028c52b30b6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:37.637740 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.638368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.639900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1285/2549] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:37 2022] DELETE /identity/v3/projects/7e11442ca5d44c3398b028c52b30b6cf => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:37.643878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:37.653445 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95904f6-4faa-4af1-b044-c5fa2266537c service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:37.708450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:37.709113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:37.709394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:37.709670 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] PATH_INFO: `/v3/users/d5885b770193431da1a0e1e33857bde1` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:37.710836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5885b770193431da1a0e1e33857bde1)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:37.712786 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:37.769906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5885b770193431da1a0e1e33857bde1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.779865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5885b770193431da1a0e1e33857bde1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.793138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5885b770193431da1a0e1e33857bde1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.793912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5885b770193431da1a0e1e33857bde1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.810257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5885b770193431da1a0e1e33857bde1'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:37.816922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1265/2550] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:37 2022] DELETE /identity/v3/users/d5885b770193431da1a0e1e33857bde1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:38.104902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.106056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.128423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2f58a-599a-4aa5-bd85-73ccac9069d7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.140020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.141468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.161475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:38.170441 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:38.190891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.192068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.192351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.192622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] PATH_INFO: `/v3/projects/7e6119cfb913409dad86a87cbec8567e` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.193760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6119cfb913409dad86a87cbec8567e)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.195721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.204382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20c43d-6c2b-48c6-a2e2-7afd1c206035 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.208598 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.209212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.209496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.209772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.211533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.244409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6119cfb913409dad86a87cbec8567e', 'request_id': 'req-472201ad-4b33-4b95-95d9-9f935bf1c6cf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:38.249855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6119cfb913409dad86a87cbec8567e', 'request_id': 'req-472201ad-4b33-4b95-95d9-9f935bf1c6cf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:38.256517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6119cfb913409dad86a87cbec8567e', 'request_id': 'req-472201ad-4b33-4b95-95d9-9f935bf1c6cf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:38.268951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] The token cache is being invalidate because project 7e6119cfb913409dad86a87cbec8567e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:38.269850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:38.270448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-472201ad-4b33-4b95-95d9-9f935bf1c6cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:38.272205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1286/2551] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:38 2022] DELETE /identity/v3/projects/7e6119cfb913409dad86a87cbec8567e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:38.276557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.276696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.277118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.277118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.278153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc9fc5-ccb0-48f7-98a0-5d468e2d6bf8 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.288618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1266/2552] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:38:38.296072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:38.297267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.360969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1287/2553] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:38:38.368969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.369398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.369712 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.390926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:38.392166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.400711 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1267/2554] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:38.405213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.415027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6c634-dad9-4c61-8cef-6ed5ac8faa1d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.419685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.420283 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.420570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.420826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.421463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:38.421763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.423470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.435294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1288/2555] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:38.440071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.450408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b153ba-7b92-4bf2-babd-4088d2a7f038 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.455275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.455877 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.456164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.456439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.457699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.459624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.490686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1268/2556] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:38 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:38.492221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.502245 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e1d2-ea59-4163-b976-ee3082a24c48 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.553814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:38.560826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.561528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.562385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.562671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.563480 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.566665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.587379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1289/2557] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:38 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:38:38.591820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.602537 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639bf21-2adf-4930-91ab-d8698ebc97f3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.607162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.607763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.608041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.608311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.608948 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.610782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.624584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1269/2558] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:38:38.629577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.641192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106427d6-7d2d-403e-a445-bf46c5c9b338 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.646736 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77462926-e186-4a54-a486-1ec668269ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.647458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462926-e186-4a54-a486-1ec668269ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.647739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462926-e186-4a54-a486-1ec668269ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.648014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77462926-e186-4a54-a486-1ec668269ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.648666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462926-e186-4a54-a486-1ec668269ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.650560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77462926-e186-4a54-a486-1ec668269ab2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.663419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1290/2559] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:38.667210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.679605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec509ad-f4c6-4e9d-9660-618f5c0a2774 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.684295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.684941 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.685219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.685490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] PATH_INFO: `/v3/projects/7ae69491ba3e4d62a8a5788891db123b/users/8e6b310928984de9a5c7e1b304fb0e1f/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.701657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ae69491ba3e4d62a8a5788891db123b, user_id=8e6b310928984de9a5c7e1b304fb0e1f, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.704380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.718405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1270/2560] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:38 2022] PUT /identity/v3/projects/7ae69491ba3e4d62a8a5788891db123b/users/8e6b310928984de9a5c7e1b304fb0e1f/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:38.728933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.729220 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.729617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.757995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] MFA Rules not processed for user `8e6b310928984de9a5c7e1b304fb0e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:38.759392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.834975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1291/2561] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:38.851637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:38.853328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:38.935046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f8f35-ad78-45c8-8f90-d8175928076d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:38.938981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca2f9d-fc2b-476e-be0d-924f7062ed33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:38.939743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2f9d-fc2b-476e-be0d-924f7062ed33 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:38.940023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2f9d-fc2b-476e-be0d-924f7062ed33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:38.940296 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca2f9d-fc2b-476e-be0d-924f7062ed33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:38.949377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca2f9d-fc2b-476e-be0d-924f7062ed33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:38.950799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca2f9d-fc2b-476e-be0d-924f7062ed33 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:38.960681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1271/2562] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:38.982648 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:39.052053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383d072-7481-451b-908b-1cb1a7cb7d9c None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:39.056871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:39.057587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:39.057940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:39.058286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] PATH_INFO: `/v3/users/8e6b310928984de9a5c7e1b304fb0e1f` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:39.059868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6b310928984de9a5c7e1b304fb0e1f)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:39.062271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:39.114637 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6b310928984de9a5c7e1b304fb0e1f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.124595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6b310928984de9a5c7e1b304fb0e1f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.125234 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6b310928984de9a5c7e1b304fb0e1f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.134683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6b310928984de9a5c7e1b304fb0e1f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.139196 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:39.149949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6b310928984de9a5c7e1b304fb0e1f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.159123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1292/2563] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:38 2022] DELETE /identity/v3/users/8e6b310928984de9a5c7e1b304fb0e1f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:39.163899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:39.164183 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:39.164462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:39.172530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:39.185975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:39.187660 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:39.261998 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1272/2564] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:39.273061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:39.274731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:39.340073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82703c-d8b5-4e05-b87b-cd39c884b318 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:39.343788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ed717-dff3-4a75-a167-485ad33ca5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:39.344420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ed717-dff3-4a75-a167-485ad33ca5ab service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:39.344702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ed717-dff3-4a75-a167-485ad33ca5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:39.344978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ed717-dff3-4a75-a167-485ad33ca5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:39.354640 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ed717-dff3-4a75-a167-485ad33ca5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:39.356101 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ed717-dff3-4a75-a167-485ad33ca5ab service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:39.365561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1293/2565] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:39.804483 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:39.806484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:39.818960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5e5ee-60a8-4f0c-bf9f-31b0fcf1cd9e None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:39.823545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:39.824260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:39.824552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:39.824819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] PATH_INFO: `/v3/projects/7ae69491ba3e4d62a8a5788891db123b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:39.825947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ae69491ba3e4d62a8a5788891db123b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:39.827955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:39.880716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ae69491ba3e4d62a8a5788891db123b', 'request_id': 'req-7c8eca07-7a30-4c39-a921-f27acd1c85d6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.886910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ae69491ba3e4d62a8a5788891db123b', 'request_id': 'req-7c8eca07-7a30-4c39-a921-f27acd1c85d6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.893620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ae69491ba3e4d62a8a5788891db123b', 'request_id': 'req-7c8eca07-7a30-4c39-a921-f27acd1c85d6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.908697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] The token cache is being invalidate because project 7ae69491ba3e4d62a8a5788891db123b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:39.908958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.909543 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7c8eca07-7a30-4c39-a921-f27acd1c85d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:39.911342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1273/2566] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:39 2022] DELETE /identity/v3/projects/7ae69491ba3e4d62a8a5788891db123b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:39.916082 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:39.916565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:39.916690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:39.936492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:39.937719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:40.000019 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1294/2567] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:38:40.009398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.009840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.009963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.040097 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:40.042219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:40.057928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.063401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1274/2568] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:40.070706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226651ea-c68f-4573-a563-3bb0fd773522 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.077260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.078474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.079003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.079313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.079977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:40.080336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.082023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.099891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1295/2569] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:40.100620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.115501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca5dd3-0ec7-4ebf-884c-f3722af6a121 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.124431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.125574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.126673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.127560 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.128439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.131659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.170198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1275/2570] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:40 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:40.174063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.185073 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0ff65-0b0d-42f5-90c5-a677938df9e4 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.251566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.252301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.252647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.252994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.254248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.256291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.281948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1296/2571] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:40 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:40.286350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.297618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4377c34-ceef-4827-b2c8-2c1607c71b9f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.305156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.306009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.306462 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.306893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.307813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.309599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.322716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1276/2572] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:40.326422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.333412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:40.349826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c3aaa-87c0-40b6-9ac0-a197714ae811 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.359924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f1798-288c-4e9f-b8b2-74396f2f56f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.360612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f1798-288c-4e9f-b8b2-74396f2f56f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.360949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f1798-288c-4e9f-b8b2-74396f2f56f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.361286 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f1798-288c-4e9f-b8b2-74396f2f56f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.361947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f1798-288c-4e9f-b8b2-74396f2f56f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.364132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f1798-288c-4e9f-b8b2-74396f2f56f2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.380203 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1297/2573] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:40.381039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.391432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3accda0d-c0f0-4e1e-8805-0c1a43bb9247 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.398984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.399897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.400336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.401199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] PATH_INFO: `/v3/projects/c0b03f429bf84009ae8827a2459c2d8f/users/99c33899c7084e97ab5e3ef61009fcdf/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.419891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0b03f429bf84009ae8827a2459c2d8f, user_id=99c33899c7084e97ab5e3ef61009fcdf, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.422639 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.438921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1277/2574] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:40 2022] PUT /identity/v3/projects/c0b03f429bf84009ae8827a2459c2d8f/users/99c33899c7084e97ab5e3ef61009fcdf/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:40.448868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.451113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.451113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.489573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] MFA Rules not processed for user `99c33899c7084e97ab5e3ef61009fcdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:40.491310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:40.580458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1298/2575] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:40.595397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:40.597790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:40.640211 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:40.688945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5214143-8b69-41e7-bd4e-3895e066eacf admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:40.693529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:40.694247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:40.694975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:40.694975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:40.705380 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:40.706597 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:40.716408 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1278/2576] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:48.865831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:48.867909 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:48.924916 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129648-3396-41fb-a8a8-0cf16b270eea None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:48.928514 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:48.929111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:48.929389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:48.930171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:48.931900 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:48.993962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:48.995445 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:49.005584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1299/2577] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:50.715950 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=15 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:52.579955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:52.581893 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:52.597295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77ca3e-eb60-4397-9bff-094fb000a87e service glance] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:52.602461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:52.603102 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:52.603401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:52.603680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] PATH_INFO: `/v3/users/48a91f6797cc4604ba2606e0549234d0` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:52.604819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48a91f6797cc4604ba2606e0549234d0)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:52.606741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:52.654553 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48a91f6797cc4604ba2606e0549234d0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:52.662139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48a91f6797cc4604ba2606e0549234d0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:52.669573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48a91f6797cc4604ba2606e0549234d0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:52.670248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48a91f6797cc4604ba2606e0549234d0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:52.683978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e44486db-8068-4c04-801c-6ffb7e311dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48a91f6797cc4604ba2606e0549234d0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:52.695245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1279/2578] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:52 2022] DELETE /identity/v3/users/48a91f6797cc4604ba2606e0549234d0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:52.886176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:52.896120 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898682d5-3a5c-406b-af01-22d545aacdbc service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:52.928139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:52.956482 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:52.957118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:52.957398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:52.957675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] PATH_INFO: `/v3/projects/cfc86120e53145328dd430f542e73182` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:52.958830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc86120e53145328dd430f542e73182)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:52.961302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.006735 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc86120e53145328dd430f542e73182', 'request_id': 'req-d88efbda-d94e-4600-ba5e-8deb0adf44c6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:53.012832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc86120e53145328dd430f542e73182', 'request_id': 'req-d88efbda-d94e-4600-ba5e-8deb0adf44c6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:53.018837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc86120e53145328dd430f542e73182', 'request_id': 'req-d88efbda-d94e-4600-ba5e-8deb0adf44c6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:53.031134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] The token cache is being invalidate because project cfc86120e53145328dd430f542e73182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:53.031429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:53.032050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-d88efbda-d94e-4600-ba5e-8deb0adf44c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:53.033505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1300/2579] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:52 2022] DELETE /identity/v3/projects/cfc86120e53145328dd430f542e73182 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:53.038632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.038958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.039265 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.059412 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:53.060627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:53.118503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1280/2580] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:38:53.126545 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.126830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.127128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.146733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:53.147932 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:53.156270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1301/2581] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:53.160573 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.170174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ce413-9d8e-4cad-999f-df3a82e8a2de None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.174789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.175425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.175705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.175982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.176631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:53.176939 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.178529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.189859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1281/2582] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:53.194051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.203818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a6860-68f5-4307-a2a5-a4d18d551805 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.208332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.208911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.209187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.209452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.210057 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.212568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.239518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1302/2583] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:53.243628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.253425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb03d4-f6fe-45ba-8e4c-bd41923874b8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.307937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.308526 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.308803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.309071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.309686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.311617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.319214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:53.334205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1282/2584] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:53.339006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.349541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d782b-54a6-4858-9d0f-e9875ca92b73 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.354108 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.354696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.354975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.355287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.355935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.357551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.367786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1303/2585] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:53.371623 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.383215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8443d47-a682-4f14-8ce4-243fbe5ebc54 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.387866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8edcc-62e7-4c98-abf1-83a1b27a2641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.388454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8edcc-62e7-4c98-abf1-83a1b27a2641 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.388736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8edcc-62e7-4c98-abf1-83a1b27a2641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.389010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8edcc-62e7-4c98-abf1-83a1b27a2641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.389649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8edcc-62e7-4c98-abf1-83a1b27a2641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.391257 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8edcc-62e7-4c98-abf1-83a1b27a2641 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.401584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1283/2586] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:53.405465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.415962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aebb8e-bfac-4152-b841-02ed0392d87a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.420655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.421403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.421752 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.422155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] PATH_INFO: `/v3/projects/6b45178df76745ec8bbbb91a0b616227/users/79753cb65ab5473a9ba3f3bdfae15a13/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.441385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b45178df76745ec8bbbb91a0b616227, user_id=79753cb65ab5473a9ba3f3bdfae15a13, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.444182 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.460013 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1304/2587] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:53 2022] PUT /identity/v3/projects/6b45178df76745ec8bbbb91a0b616227/users/79753cb65ab5473a9ba3f3bdfae15a13/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:53.467953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.468244 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.468521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.497960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] MFA Rules not processed for user `79753cb65ab5473a9ba3f3bdfae15a13`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:53.499504 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:53.568193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1284/2588] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:53.573565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.637872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f040d-056c-49dc-88e8-d3bb96d5e391 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.642567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.643205 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.643489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.643763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.644383 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.646292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.650061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:53.697319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1305/2589] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/projects => generated 401 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:53.701482 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.752952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943d75f-5adb-4963-ab64-c62aae47b944 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.812078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.812727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.813012 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.813291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.813949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.815899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.837618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1285/2590] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:53 2022] POST /identity/v3/users => generated 307 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:38:53.841622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.889855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ac0b-5420-4e81-ad65-61d40323b238 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.894499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.895125 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.895426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.895692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.896327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.897903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.907542 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1306/2591] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:53.911829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.922314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9f0b8-a0c1-49b6-8c5b-5bc8e4c56e71 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:53.926816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:53.927494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:53.927772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:53.928045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] PATH_INFO: `/v3/projects/9f3242bc2df84adfa43dc50a943462f9/users/7f6d6aea1e314daaa135939eb460d521/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:53.942617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3242bc2df84adfa43dc50a943462f9, user_id=7f6d6aea1e314daaa135939eb460d521, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:53.945820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:53.947958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:53.959423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1286/2592] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:53 2022] PUT /identity/v3/projects/9f3242bc2df84adfa43dc50a943462f9/users/7f6d6aea1e314daaa135939eb460d521/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:53.963860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:53.973690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b6f8-3329-407a-b678-323a4da24dcb admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:54.027497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a910b91-5160-480b-9fd0-a5b7c4c49cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:54.028020 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b91-5160-480b-9fd0-a5b7c4c49cab admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:54.028296 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b91-5160-480b-9fd0-a5b7c4c49cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:54.028564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a910b91-5160-480b-9fd0-a5b7c4c49cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:54.029189 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a910b91-5160-480b-9fd0-a5b7c4c49cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:54.030791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a910b91-5160-480b-9fd0-a5b7c4c49cab admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:54.040481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1307/2593] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:54.044675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:54.055015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f65eb-276f-434d-929d-3245f9974fce admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:54.059574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:54.060216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:54.060497 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:54.060768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] PATH_INFO: `/v3/projects/9f3242bc2df84adfa43dc50a943462f9/users/7f6d6aea1e314daaa135939eb460d521/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:54.062676 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3242bc2df84adfa43dc50a943462f9, user_id=7f6d6aea1e314daaa135939eb460d521, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:54.065328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:54.076061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1287/2594] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:54 2022] PUT /identity/v3/projects/9f3242bc2df84adfa43dc50a943462f9/users/7f6d6aea1e314daaa135939eb460d521/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:54.088300 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:54.088590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:54.088869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:54.115499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] MFA Rules not processed for user `7f6d6aea1e314daaa135939eb460d521`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:54.116700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:54.182788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1308/2595] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:54 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:54.193283 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:54.194994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:54.268202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dda804-834f-4d1b-804e-df1d22d0b03e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:54.271914 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:54.272518 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:54.272793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:54.273594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:54.282575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:54.283972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:54.294099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1288/2596] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:54 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:54.365943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:54.370511 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:54.377227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0425d-44c5-4dbe-b329-f7cf538fe079 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:54.381142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:54.381742 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:54.382025 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:54.382298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:54.439285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:54.440721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:54.449615 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1309/2597] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:54.709096 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:54.711111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:54.773860 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9ed2e-9813-4618-a9ae-dcfa0aa3dc97 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:54.777996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:54.778628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:54.778910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:54.779306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:54.781400 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:54.848965 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:54.850545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:54.861386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1289/2598] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:58.135665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:58.137106 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.188268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:58.189694 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.202335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39687173-6a6b-4cb2-b137-90500c10dc0b service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:58.208235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:58.208867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.209138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.209595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] PATH_INFO: `/v3/users/84d2f3c3c5fb4cceabf9a35f64a1551c` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.210703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d2f3c3c5fb4cceabf9a35f64a1551c)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:58.212815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:58.242643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721c1c-570f-48c9-b4f2-c0acf5348515 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:58.251044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:58.251825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.252171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.252469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.254650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.267705 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:58.269336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:58.280523 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1310/2599] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:58.352241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:58.386321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d2f3c3c5fb4cceabf9a35f64a1551c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.393055 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d2f3c3c5fb4cceabf9a35f64a1551c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.399699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d2f3c3c5fb4cceabf9a35f64a1551c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.400489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d2f3c3c5fb4cceabf9a35f64a1551c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.412579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d2f3c3c5fb4cceabf9a35f64a1551c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.418798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1290/2600] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:58 2022] DELETE /identity/v3/users/84d2f3c3c5fb4cceabf9a35f64a1551c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:38:58.545890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:58.547355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.563075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3364a81-fee0-4940-a135-0a5bb046bd0a service placement] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:58.611648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:58.613274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.649550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:58.650621 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.651066 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.651556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] PATH_INFO: `/v3/users/99c33899c7084e97ab5e3ef61009fcdf` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.652851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99c33899c7084e97ab5e3ef61009fcdf)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:58.655607 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:58.671141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:58.694891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8c30-00f3-44af-8e2f-d05ea265dde1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:58.703984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:58.705121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.705403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.705705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] PATH_INFO: `/v3/users/79753cb65ab5473a9ba3f3bdfae15a13` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.706820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=79753cb65ab5473a9ba3f3bdfae15a13)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:58.708765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:58.709888 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99c33899c7084e97ab5e3ef61009fcdf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.720426 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99c33899c7084e97ab5e3ef61009fcdf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.721116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99c33899c7084e97ab5e3ef61009fcdf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.732633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99c33899c7084e97ab5e3ef61009fcdf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.749466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99c33899c7084e97ab5e3ef61009fcdf'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.757443 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1311/2601] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:58 2022] DELETE /identity/v3/users/99c33899c7084e97ab5e3ef61009fcdf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:58.761084 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:58.763364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.766435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79753cb65ab5473a9ba3f3bdfae15a13'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.775627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79753cb65ab5473a9ba3f3bdfae15a13'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.796187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79753cb65ab5473a9ba3f3bdfae15a13'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.796892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79753cb65ab5473a9ba3f3bdfae15a13'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.814271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f04bb6-3cc2-4617-be6a-af1900b9a5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79753cb65ab5473a9ba3f3bdfae15a13'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.822264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1291/2602] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:58 2022] DELETE /identity/v3/users/79753cb65ab5473a9ba3f3bdfae15a13 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:58.823677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f900c-fd67-4d28-b33a-406a3ab8cdf2 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.823880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f900c-fd67-4d28-b33a-406a3ab8cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.824149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f900c-fd67-4d28-b33a-406a3ab8cdf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.842805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3568974b-f4de-4390-b3ce-6193502a9f75 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:58.845273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:58.854767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-a66f900c-fd67-4d28-b33a-406a3ab8cdf2 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:58.856124 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a66f900c-fd67-4d28-b33a-406a3ab8cdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.906790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:58.907465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.907778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.908058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] PATH_INFO: `/v3/projects/fa2ecaeb6faf4c1ea5eac70b19398b42` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.909247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2ecaeb6faf4c1ea5eac70b19398b42)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:58.911495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:58.917139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1292/2603] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:58.918375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:58.918663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:58.918940 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:58.938190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:58.939495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.947607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1293/2604] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:58.948378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:58.949635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:58.963310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:58.975495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2ecaeb6faf4c1ea5eac70b19398b42', 'request_id': 'req-fd785268-e113-4374-a55d-a18d8ac8ba7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.982242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2ecaeb6faf4c1ea5eac70b19398b42', 'request_id': 'req-fd785268-e113-4374-a55d-a18d8ac8ba7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:58.988567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2ecaeb6faf4c1ea5eac70b19398b42', 'request_id': 'req-fd785268-e113-4374-a55d-a18d8ac8ba7a'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.001204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] The token cache is being invalidate because project fa2ecaeb6faf4c1ea5eac70b19398b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:59.001502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.002144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.003697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1312/2605] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:58 2022] DELETE /identity/v3/projects/fa2ecaeb6faf4c1ea5eac70b19398b42 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:59.008336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.009662 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.009778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58931a-8f13-4561-a468-d2dc60c72ac3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.058778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.059410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.059691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.059964 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.061738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.067065 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd785268-e113-4374-a55d-a18d8ac8ba7a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.071516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.072119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.072409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.072678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.074413 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.124587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.126028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8522bfc9-f348-421c-8160-586555781f62 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.135932 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1294/2606] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:59.137115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.137398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.137725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.138836 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.140317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039cf8e-dcdd-4d82-abda-f9803b4d761c service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.142072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:59.150995 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1313/2607] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:38:59.193347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:59.194487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.203888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1295/2608] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:59.212653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.212948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.213356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.235893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:59.237187 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.246195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1314/2609] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:59.251060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.263199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7380b17-59fb-47f6-8a73-d61cdcbcd529 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.268729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.269421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.270151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.270151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.270741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:59.271138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.272872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.287134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1296/2610] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:59.291333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.312074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14aebd-96b6-4fb3-9132-1977c21eb82b None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.330883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.331825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.332436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.332715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.333438 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.336553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.385061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1315/2611] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:59.389261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.402540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fbc46-f86e-430b-8deb-c52989f47869 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.452009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.454491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.483009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.483668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.483984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.484321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.484997 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.487194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.511275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1297/2612] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:38:59.512222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.513919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.525566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f20fa-bf3e-4dc8-baf6-ae00de20091d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.530823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.531574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.531708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:59.531979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.532326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] PATH_INFO: `/v3/projects/c0b03f429bf84009ae8827a2459c2d8f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.533768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0b03f429bf84009ae8827a2459c2d8f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.535925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.548533 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22867c-700b-4556-a273-77909874e763 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.553490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.554642 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.554982 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.555292 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] PATH_INFO: `/v3/projects/6b45178df76745ec8bbbb91a0b616227` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.556444 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b45178df76745ec8bbbb91a0b616227)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.558407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.585369 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0b03f429bf84009ae8827a2459c2d8f', 'request_id': 'req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.591654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0b03f429bf84009ae8827a2459c2d8f', 'request_id': 'req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.599225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0b03f429bf84009ae8827a2459c2d8f', 'request_id': 'req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.607994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b45178df76745ec8bbbb91a0b616227', 'request_id': 'req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.611760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] The token cache is being invalidate because project c0b03f429bf84009ae8827a2459c2d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:59.612058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.612683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.614034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b45178df76745ec8bbbb91a0b616227', 'request_id': 'req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.614216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1298/2613] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:59 2022] DELETE /identity/v3/projects/c0b03f429bf84009ae8827a2459c2d8f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:59.615071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.616439 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.622370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b45178df76745ec8bbbb91a0b616227', 'request_id': 'req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.637711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] The token cache is being invalidate because project 6b45178df76745ec8bbbb91a0b616227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:38:59.638351 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.638695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-389f95bc-3a5d-4b94-9d3b-46b36aef8f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.640626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1316/2614] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:38:59 2022] DELETE /identity/v3/projects/6b45178df76745ec8bbbb91a0b616227 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:59.641763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.642051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.642344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.652501 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:59.664332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:59.665537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.686194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ea7a-f442-4b97-9a73-26eb585e1f85 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.731164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1317/2615] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:59.731164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.731164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.738468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610cf94-b27d-4032-80c8-c0fe300ce392 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.743568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.743736 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996df1a8-7494-481c-9f50-b2f2db3a5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.744178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.744339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996df1a8-7494-481c-9f50-b2f2db3a5783 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.744452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.744549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996df1a8-7494-481c-9f50-b2f2db3a5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.744706 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] PATH_INFO: `/v3/users/7f6d6aea1e314daaa135939eb460d521` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.745015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996df1a8-7494-481c-9f50-b2f2db3a5783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.745688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996df1a8-7494-481c-9f50-b2f2db3a5783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.745802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6d6aea1e314daaa135939eb460d521)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.747372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996df1a8-7494-481c-9f50-b2f2db3a5783 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.747727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.758121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1299/2616] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:59.759419 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.759715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.760288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.779635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:38:59.781282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.790903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1300/2617] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:38:59.791715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.794499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.799824 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6d6aea1e314daaa135939eb460d521'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.806511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96f2d0-f64d-45e5-8c30-c1febc18b868 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.808012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6d6aea1e314daaa135939eb460d521'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.808663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6d6aea1e314daaa135939eb460d521'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.811306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.811897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.812192 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.812437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.813200 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.814822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.816031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6d6aea1e314daaa135939eb460d521'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.824053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:38:59.828530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1301/2618] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:38:59.828530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.828530 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.833666 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6d6aea1e314daaa135939eb460d521'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:38:59.840049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1318/2619] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:38:59 2022] DELETE /identity/v3/users/7f6d6aea1e314daaa135939eb460d521 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:59.840398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541da4e2-5a88-4cce-9c28-c093e9bdd841 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.840805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.842126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:38:59.918502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.919454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.920029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.920029 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.921270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:38:59.921586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.923421 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.924261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28788b2e-1e4b-4dd3-b9c2-e485f42aab1c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.932658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6be7e-c2f9-4f21-909b-7b7ac7dc6e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.934471 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6be7e-c2f9-4f21-909b-7b7ac7dc6e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.934753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6be7e-c2f9-4f21-909b-7b7ac7dc6e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.935031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6be7e-c2f9-4f21-909b-7b7ac7dc6e07 admin admin] PATH_INFO: `/v3/projects/b551f48180454c26b4d95161bac3c737/users/9487a89623254255aec576574d400bdf/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.937478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1302/2620] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:38:59.941516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:38:59.951044 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6be7e-c2f9-4f21-909b-7b7ac7dc6e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b551f48180454c26b4d95161bac3c737, user_id=9487a89623254255aec576574d400bdf, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.952987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9902c576-75ba-47b0-bf4c-95b8f86ea960 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:38:59.954457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6be7e-c2f9-4f21-909b-7b7ac7dc6e07 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.959365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:38:59.960237 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.960629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.960985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:38:59.961677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:38:59.963687 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:38:59.971048 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1319/2621] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:38:59 2022] PUT /identity/v3/projects/b551f48180454c26b4d95161bac3c737/users/9487a89623254255aec576574d400bdf/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:38:59.977258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:38:59.977549 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:38:59.977826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.003987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1303/2622] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:00.004211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:00.005540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.008071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.044390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:00.079128 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1320/2623] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:00.081132 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc392870-c389-4e79-9422-cbaa8f73b421 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.089403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.090473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.091118 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.091399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.091631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.091742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.092301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.094260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.120992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1304/2624] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:00.125197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.142775 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58e5c9-ca41-4be2-956d-4a62947d62fe admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.149260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.149872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.150151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.150450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.151113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.152717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.165000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1305/2625] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:00.168844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.170629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.172652 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a8af2-a90e-455d-946b-24aa778ab101 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.176667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.177280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.177592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.177860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.187716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.189153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.190910 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415b345-0bac-440c-94c1-51ba9720a67e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.195839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.196461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.196742 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.197020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] PATH_INFO: `/v3/projects/9f3242bc2df84adfa43dc50a943462f9` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.198139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3242bc2df84adfa43dc50a943462f9)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.198885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1321/2626] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:39:00.199725 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.200152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.208881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037250-44e9-4659-be29-bfff98d5390c service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.215006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.215348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.215632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.215944 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.216547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.218155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.230359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1322/2627] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:00.234582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.244891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3042e-df3b-4734-aeb2-674370a70e47 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.248789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:00.254481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3242bc2df84adfa43dc50a943462f9', 'request_id': 'req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:00.260272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3242bc2df84adfa43dc50a943462f9', 'request_id': 'req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:00.267039 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3242bc2df84adfa43dc50a943462f9', 'request_id': 'req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:00.280713 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] The token cache is being invalidate because project 9f3242bc2df84adfa43dc50a943462f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:00.281006 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:00.282028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-319e197b-0ea7-4f3b-a8cd-b90f0481cb1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:00.287988 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1306/2628] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:00 2022] DELETE /identity/v3/projects/9f3242bc2df84adfa43dc50a943462f9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:00.290144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b35cb-9cfd-439f-a650-4da550db3f33 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.290431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b35cb-9cfd-439f-a650-4da550db3f33 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.290700 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b35cb-9cfd-439f-a650-4da550db3f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.313242 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-435b35cb-9cfd-439f-a650-4da550db3f33 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:00.314654 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-435b35cb-9cfd-439f-a650-4da550db3f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.315219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30903eb-801a-4464-80ef-5cb537dfae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.315802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30903eb-801a-4464-80ef-5cb537dfae71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.316086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30903eb-801a-4464-80ef-5cb537dfae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.316417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30903eb-801a-4464-80ef-5cb537dfae71 admin admin] PATH_INFO: `/v3/projects/c8cf1a22c29b4cdfb1e24ae72aa70d9b/users/867a9dc5842e4159b8f4930358635e1c/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.323621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1307/2629] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:00.331757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.332095 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.332393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.333833 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30903eb-801a-4464-80ef-5cb537dfae71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cf1a22c29b4cdfb1e24ae72aa70d9b, user_id=867a9dc5842e4159b8f4930358635e1c, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.337497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30903eb-801a-4464-80ef-5cb537dfae71 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.354072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1323/2630] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:00 2022] PUT /identity/v3/projects/c8cf1a22c29b4cdfb1e24ae72aa70d9b/users/867a9dc5842e4159b8f4930358635e1c/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:00.354389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:00.356223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.365520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.365936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.366221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.400722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] MFA Rules not processed for user `867a9dc5842e4159b8f4930358635e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:00.402121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.442566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1308/2631] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:00.446693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.457866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258318-3aab-4c58-a83a-60eb6491dbfd None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.463357 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.463960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.464339 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.464803 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.465459 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:00.465882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.467802 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.476703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1324/2632] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:00.482824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1309/2633] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:39:00.487076 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.492172 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.495944 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.499696 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe253b5f-af6d-4f7a-80cd-06c9a5a19875 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.504318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.504979 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.505265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.505540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.506290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.508814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.511476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:00.550509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1325/2634] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:00.555622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.591652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edabe4-d61e-46b5-8b60-5bfe185f14d1 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.636124 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:00.641356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d59e-d901-407c-999c-9d734097e583 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.649462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.650267 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.650626 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.650953 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.651667 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.653714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.669823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca55c3-5e59-40aa-b824-569ee5d74bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.670466 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca55c3-5e59-40aa-b824-569ee5d74bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.670795 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca55c3-5e59-40aa-b824-569ee5d74bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.671156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca55c3-5e59-40aa-b824-569ee5d74bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.680279 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1326/2635] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:00.683949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.694843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cab191-323c-4bc7-8a77-47efbfee19cd admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.701290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.702201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.702487 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.702761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.703474 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.705116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.718395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1327/2636] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:00.722484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.732828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5a284-632b-4482-bdf0-7b17e32850b7 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.737528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.738123 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.738418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.738689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.739366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.740967 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.753743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1328/2637] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:00.757894 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca55c3-5e59-40aa-b824-569ee5d74bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.758294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.759469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ca55c3-5e59-40aa-b824-569ee5d74bb6 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.768878 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed0700-83ca-4d1a-b5f5-5eb0bed81482 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:00.771157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1310/2638] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:39:00.775579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572d5a1-768c-4490-a571-fa44cee1c797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:00.776322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572d5a1-768c-4490-a571-fa44cee1c797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.776797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572d5a1-768c-4490-a571-fa44cee1c797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.776972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572d5a1-768c-4490-a571-fa44cee1c797 admin admin] PATH_INFO: `/v3/projects/c189d92065934d928af13f4f22257a6d/users/3614aa5652614d9381af21acdc18fae6/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.794806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572d5a1-768c-4490-a571-fa44cee1c797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c189d92065934d928af13f4f22257a6d, user_id=3614aa5652614d9381af21acdc18fae6, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:00.802708 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572d5a1-768c-4490-a571-fa44cee1c797 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:00.827176 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1329/2639] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:00 2022] PUT /identity/v3/projects/c189d92065934d928af13f4f22257a6d/users/3614aa5652614d9381af21acdc18fae6/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:00.844773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.845063 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.845337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.880450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] MFA Rules not processed for user `3614aa5652614d9381af21acdc18fae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:00.881750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.927809 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:00.928093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:00.928370 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:00.973682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1311/2640] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:00.976436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:00.977080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] MFA Rules not processed for user `9487a89623254255aec576574d400bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:00.978376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:00.994771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:01.062919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcc7d2-2fec-4430-b087-389cfdd5cce5 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.067990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.069308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.069709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.070702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.071740 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.074053 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.082656 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1330/2641] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:01.083757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.085779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:01.102349 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08361d-563f-4ae2-b6d9-f7ac17b20612 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.108758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1312/2642] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:00 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:01.111496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.112476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.113595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.114110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.114605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.115556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.119625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.167230 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1331/2643] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:01 2022] POST /identity/v3/projects => generated 405 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:01.171083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.212465 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaa33e-608d-45c2-b0a4-f735d739dab9 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.262463 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b60e8-c397-4e8e-a7a8-a6526fd8f747 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.267718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.268500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.268778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.269049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.269689 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.271631 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.272450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.273106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.274003 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.274288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.274916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.276870 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.288165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:01.296420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1332/2644] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:01 2022] POST /identity/v3/users => generated 309 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:01.300083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.300695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1313/2645] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:01 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:01.304750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.311532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426050e-2a02-4799-a7f7-53831cb62c17 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.314234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96bf7b-2bcd-4da3-802e-0c0d6b92b30e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.316397 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.317171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.317483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.317796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.318449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.319008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.319655 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.319935 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.320068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.320335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.320975 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.322725 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.330038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1333/2646] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:01.333091 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1314/2647] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:01.334219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.336788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.343904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e1df8-2476-41ed-a1ab-ab922296b42b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.346267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e041679-2b62-46d1-a321-9e7aabc5be58 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.348548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.349212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.349495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.349769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] PATH_INFO: `/v3/projects/1277cef79b39457294d164d9b4f20f9d/users/3251efa387e14a509148b1def51ee951/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.351490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.352109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.352499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.352773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] PATH_INFO: `/v3/projects/b47cf6519f984f62b88fb7e9be869b7b/users/236200a994354560b3810f85854db8f8/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.364813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1277cef79b39457294d164d9b4f20f9d, user_id=3251efa387e14a509148b1def51ee951, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.367824 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47cf6519f984f62b88fb7e9be869b7b, user_id=236200a994354560b3810f85854db8f8, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.368103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.370484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.378614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:01.383389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1334/2648] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:01 2022] PUT /identity/v3/projects/1277cef79b39457294d164d9b4f20f9d/users/3251efa387e14a509148b1def51ee951/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:01.386041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1315/2649] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:01 2022] PUT /identity/v3/projects/b47cf6519f984f62b88fb7e9be869b7b/users/236200a994354560b3810f85854db8f8/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:01.388024 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.389943 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.453054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20995298-6f5f-4399-aafe-15e76cd04da2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.453918 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b1ba1-dd53-4890-8c59-571b41a09793 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.458979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.459681 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.459956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.460063 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.460263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.460670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.460913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.461033 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.461202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.461928 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.462510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.463567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.473579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1316/2650] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:01.475792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1335/2651] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:01.480088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.480215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.489842 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d67b7-4e52-4a11-937c-9381b1c283a0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.490287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6aaa0-f515-488e-80b1-d8a7b896d439 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.494969 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c250a6-4176-4b4c-b02f-34f9134f5270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.495578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.495720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c250a6-4176-4b4c-b02f-34f9134f5270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.495898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c250a6-4176-4b4c-b02f-34f9134f5270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.496169 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c250a6-4176-4b4c-b02f-34f9134f5270 admin admin] PATH_INFO: `/v3/projects/1277cef79b39457294d164d9b4f20f9d/users/3251efa387e14a509148b1def51ee951/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.496347 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.496634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.496906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] PATH_INFO: `/v3/projects/b47cf6519f984f62b88fb7e9be869b7b/users/236200a994354560b3810f85854db8f8/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.498104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c250a6-4176-4b4c-b02f-34f9134f5270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1277cef79b39457294d164d9b4f20f9d, user_id=3251efa387e14a509148b1def51ee951, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.498806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47cf6519f984f62b88fb7e9be869b7b, user_id=236200a994354560b3810f85854db8f8, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.500741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c250a6-4176-4b4c-b02f-34f9134f5270 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.501480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.511141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1336/2652] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:01 2022] PUT /identity/v3/projects/1277cef79b39457294d164d9b4f20f9d/users/3251efa387e14a509148b1def51ee951/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:01.511770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1317/2653] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:01 2022] PUT /identity/v3/projects/b47cf6519f984f62b88fb7e9be869b7b/users/236200a994354560b3810f85854db8f8/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:01.521516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.521829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.522104 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.550385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] MFA Rules not processed for user `236200a994354560b3810f85854db8f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:01.551629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:01.623748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1337/2654] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:01.635936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.702667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec86b51-a7ed-4ed3-a59e-38ebec7b0ef8 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.706255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0492f-5a78-46eb-bb8e-3f49e74b375f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.706852 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0492f-5a78-46eb-bb8e-3f49e74b375f service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.707154 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0492f-5a78-46eb-bb8e-3f49e74b375f service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.707425 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0492f-5a78-46eb-bb8e-3f49e74b375f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.717819 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0492f-5a78-46eb-bb8e-3f49e74b375f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.719309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0492f-5a78-46eb-bb8e-3f49e74b375f service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.730368 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1318/2655] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:39:01.824312 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:01.838881 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73308207-c29d-46af-8acb-5541ee0fef33 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:01.843605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:01.844499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:01.844860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:01.845100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:01.881447 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:01.916498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:01.918233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:01.929552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1338/2656] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:02.170728 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:02.171018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:02.171318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:02.223057 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] MFA Rules not processed for user `3251efa387e14a509148b1def51ee951`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:02.224915 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:02.320197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1319/2657] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:02.735874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:02.737573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:02.811346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde420-0313-4fff-a46b-d16a48dd0d07 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:02.816122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:02.816734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:02.817038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:02.817314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:02.819324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:02.831951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:02.833530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:02.850665 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1339/2658] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:03.389516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:03.401851 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:03.457580 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66538886-b1c9-4410-98ac-0e01e851c180 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:03.463017 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:03.463702 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:03.465552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:03.465858 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] PATH_INFO: `/v3/users/9487a89623254255aec576574d400bdf` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:03.467075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9487a89623254255aec576574d400bdf)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:03.468983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:03.489940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:03.491571 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:03.505559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc00b6-7408-413a-984f-04691baf0ecb service placement] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:03.511071 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:03.511721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:03.511996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:03.512336 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] PATH_INFO: `/v3/users/867a9dc5842e4159b8f4930358635e1c` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:03.513368 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=867a9dc5842e4159b8f4930358635e1c)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:03.515333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:03.525149 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9487a89623254255aec576574d400bdf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.534954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9487a89623254255aec576574d400bdf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.546243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9487a89623254255aec576574d400bdf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.547287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9487a89623254255aec576574d400bdf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.565550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f939b8fb-0ed7-4cda-b82e-0ab14f31a0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9487a89623254255aec576574d400bdf'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.574933 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1320/2659] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:03 2022] DELETE /identity/v3/users/9487a89623254255aec576574d400bdf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:03.584612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867a9dc5842e4159b8f4930358635e1c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.600687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867a9dc5842e4159b8f4930358635e1c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.601962 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867a9dc5842e4159b8f4930358635e1c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.610996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867a9dc5842e4159b8f4930358635e1c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.625826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867a9dc5842e4159b8f4930358635e1c'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.632015 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1340/2660] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:03 2022] DELETE /identity/v3/users/867a9dc5842e4159b8f4930358635e1c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:03.638409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:03.638613 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:03.638898 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:03.660023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:03.661372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:03.724416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1321/2661] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:03.734509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:03.736167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:03.772853 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:03.774498 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:03.775587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:03.788805 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ee4f9-747b-46af-8830-30162984a7b3 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:03.794333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:03.795222 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:03.795659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:03.796214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] PATH_INFO: `/v3/projects/b551f48180454c26b4d95161bac3c737` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:03.798071 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b551f48180454c26b4d95161bac3c737)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:03.800045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:03.812255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb492e68-c4f5-433a-a9b5-4a7aec4b7c87 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:03.815951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:03.816560 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:03.816839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:03.817181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:03.826820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:03.828567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:03.830260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:03.840750 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1341/2662] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:03.851848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b551f48180454c26b4d95161bac3c737', 'request_id': 'req-0d526693-fbce-40df-8056-bbfc0d145f6e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.857734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b551f48180454c26b4d95161bac3c737', 'request_id': 'req-0d526693-fbce-40df-8056-bbfc0d145f6e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.864072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b551f48180454c26b4d95161bac3c737', 'request_id': 'req-0d526693-fbce-40df-8056-bbfc0d145f6e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.878131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] The token cache is being invalidate because project b551f48180454c26b4d95161bac3c737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:03.878423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.879075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:03.880637 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1322/2663] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:03 2022] DELETE /identity/v3/projects/b551f48180454c26b4d95161bac3c737 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:03.884755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:03.895357 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be85b7-df81-43e1-85ae-ccb28c755c84 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:03.955415 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:03.956054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:03.956334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:03.956718 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] PATH_INFO: `/v3/users/3251efa387e14a509148b1def51ee951` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:03.957887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3251efa387e14a509148b1def51ee951)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:03.960771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.015074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3251efa387e14a509148b1def51ee951'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.022489 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3251efa387e14a509148b1def51ee951'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.023924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3251efa387e14a509148b1def51ee951'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.030733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3251efa387e14a509148b1def51ee951'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.043719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3251efa387e14a509148b1def51ee951'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.049572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1342/2664] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:03 2022] DELETE /identity/v3/users/3251efa387e14a509148b1def51ee951 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:04.124385 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.125888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.139799 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d526693-fbce-40df-8056-bbfc0d145f6e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.200621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.201316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.201606 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.201871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] PATH_INFO: `/v3/projects/c8cf1a22c29b4cdfb1e24ae72aa70d9b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.202982 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cf1a22c29b4cdfb1e24ae72aa70d9b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.204919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.234412 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.235846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.254360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910aab59-7034-4a4f-8fe4-0e6915fc20f5 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.263177 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cf1a22c29b4cdfb1e24ae72aa70d9b', 'request_id': 'req-f7510c68-7956-4d0b-8a1b-c581484303f9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.273453 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cf1a22c29b4cdfb1e24ae72aa70d9b', 'request_id': 'req-f7510c68-7956-4d0b-8a1b-c581484303f9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.281695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cf1a22c29b4cdfb1e24ae72aa70d9b', 'request_id': 'req-f7510c68-7956-4d0b-8a1b-c581484303f9'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.297190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] The token cache is being invalidate because project c8cf1a22c29b4cdfb1e24ae72aa70d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:04.297566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.298266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f7510c68-7956-4d0b-8a1b-c581484303f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.300245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1323/2665] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:04 2022] DELETE /identity/v3/projects/c8cf1a22c29b4cdfb1e24ae72aa70d9b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:04.304946 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128993c7-c19a-45eb-b289-24ccc9884d7e None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.305234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128993c7-c19a-45eb-b289-24ccc9884d7e None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.305507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128993c7-c19a-45eb-b289-24ccc9884d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.326945 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-128993c7-c19a-45eb-b289-24ccc9884d7e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:04.328249 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-128993c7-c19a-45eb-b289-24ccc9884d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.332849 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:04.344622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.345258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.345540 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.345841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] PATH_INFO: `/v3/projects/1277cef79b39457294d164d9b4f20f9d` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.346990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1277cef79b39457294d164d9b4f20f9d)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.348940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.368020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:04.391891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1324/2666] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:04.398633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1277cef79b39457294d164d9b4f20f9d', 'request_id': 'req-72865351-8233-4c79-aa19-6c194fc3e7b0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.400088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.400370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.400646 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.404314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1277cef79b39457294d164d9b4f20f9d', 'request_id': 'req-72865351-8233-4c79-aa19-6c194fc3e7b0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.410723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1277cef79b39457294d164d9b4f20f9d', 'request_id': 'req-72865351-8233-4c79-aa19-6c194fc3e7b0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.419949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:04.421158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.423343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] The token cache is being invalidate because project 1277cef79b39457294d164d9b4f20f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:04.423632 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.424260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72865351-8233-4c79-aa19-6c194fc3e7b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:04.425743 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1343/2667] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:04 2022] DELETE /identity/v3/projects/1277cef79b39457294d164d9b4f20f9d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:04.430573 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c941f-da7a-46ba-b47a-da0029f1e87e None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.430874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c941f-da7a-46ba-b47a-da0029f1e87e None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.431213 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c941f-da7a-46ba-b47a-da0029f1e87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.451118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-201c941f-da7a-46ba-b47a-da0029f1e87e None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:04.452345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-201c941f-da7a-46ba-b47a-da0029f1e87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.483816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1325/2668] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:04.488146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.497461 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d10b3-3c68-4319-9893-5d7b5080039d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.502273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.502873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.503174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.503454 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.504086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:04.504386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.505953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.511371 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1344/2669] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:04.521190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1326/2670] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:39:04.521361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.521361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.521361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.522114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.532147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531755ff-51dc-473b-8e01-06cde3004911 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.536854 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.537457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.537732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.538008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.538653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.539302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:04.540495 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.540748 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.548758 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1345/2671] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:04.552947 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.563054 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4134a28-38aa-4af0-9361-91cbe00f4180 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.567739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.568316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1327/2672] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:04.568459 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.568605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.568997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.570145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:04.570452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.572096 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.572758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.588355 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1346/2673] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:39:04.589088 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.599129 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b8642-4df3-4166-b94f-b6ed7da2aa5e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.642117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669695c-fbfb-4261-a3fb-1a695ce261b3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.647266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.648381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.648679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.648938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.650133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.652449 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.669699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.670173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.670505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.670837 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.671547 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.673433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.679856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1328/2674] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:04.684496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:04.687705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.700936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401de1a-2799-4562-9b07-46a5f6b12475 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.705753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.706448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.706790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.707187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.707903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.709645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.710955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1347/2675] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:04.712157 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.723135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1329/2676] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:04.725634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.736430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07519-e7fb-4f91-929d-6f94a4566f7d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.766591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:04.782794 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c360b01-fdcc-4b0c-914d-e76bcf5ed986 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.788158 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.788874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.789252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.789596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.790293 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.792529 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.812667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.813397 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.813679 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.813949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.814594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.816318 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.817225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1348/2677] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:04.821658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.827759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1330/2678] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:04.831820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.832329 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5bffd7-966d-4249-a372-dec3327544af admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.837760 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.838344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.838739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.839085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.840053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.841871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.845310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37771-5939-48af-807c-b9778f58d532 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.850072 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.850789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.851069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.851370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] PATH_INFO: `/v3/projects/ec53a2ca56b84f5dbf01c9a044ffdccd/users/447e6667a9fa4ec89e53a40c0cb008ba/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.858975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1349/2679] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 14:39:04.868678 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.884809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec53a2ca56b84f5dbf01c9a044ffdccd, user_id=447e6667a9fa4ec89e53a40c0cb008ba, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.886776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd0d97-3fc3-4967-a2a5-4aff4976b277 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:04.887490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.891714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94714-a9af-4352-b704-9588b11b6172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:04.892311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94714-a9af-4352-b704-9588b11b6172 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.892594 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94714-a9af-4352-b704-9588b11b6172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.892870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94714-a9af-4352-b704-9588b11b6172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.893524 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94714-a9af-4352-b704-9588b11b6172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:04.895266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94714-a9af-4352-b704-9588b11b6172 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:04.904545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1331/2680] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:04 2022] PUT /identity/v3/projects/ec53a2ca56b84f5dbf01c9a044ffdccd/users/447e6667a9fa4ec89e53a40c0cb008ba/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:04.906588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1350/2681] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:04.912267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:04.914746 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:04.915040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:04.915361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:04.950999 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] MFA Rules not processed for user `447e6667a9fa4ec89e53a40c0cb008ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:04.952376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:04.995755 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7f3268-28cc-4a3d-8188-148e4efbf765 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.000546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286c42c-33f1-4a2f-9d97-7a210d64c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.001301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286c42c-33f1-4a2f-9d97-7a210d64c114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.001614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286c42c-33f1-4a2f-9d97-7a210d64c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.001896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286c42c-33f1-4a2f-9d97-7a210d64c114 admin admin] PATH_INFO: `/v3/projects/2f8f5f6873e64de7a7be06a8e3577da6/users/bbb2a2cf1f4141928b0d4a20c9ca65b5/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.019610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286c42c-33f1-4a2f-9d97-7a210d64c114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8f5f6873e64de7a7be06a8e3577da6, user_id=bbb2a2cf1f4141928b0d4a20c9ca65b5, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.022243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286c42c-33f1-4a2f-9d97-7a210d64c114 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.025116 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1351/2682] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:05.039861 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1332/2683] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:04 2022] PUT /identity/v3/projects/2f8f5f6873e64de7a7be06a8e3577da6/users/bbb2a2cf1f4141928b0d4a20c9ca65b5/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:05.040898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.042568 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:05.049984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.050273 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.050581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.091041 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] MFA Rules not processed for user `bbb2a2cf1f4141928b0d4a20c9ca65b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:05.092206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:05.108624 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:05.117319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf95a3-7182-4152-9407-38a9421ffc96 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.171253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1333/2684] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:05.175628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.176310 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9355f8f80a434ab8af3145f7e82c277c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.176997 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.177465 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.177814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.191653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:05.250675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684b4f4-e025-4403-9d3d-84a3d6ecf30f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.252026 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.253464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.260772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.260772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.260772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.260772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.261158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.261158 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.263392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1352/2685] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:39:05.294133 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1334/2686] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:05 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:05.295582 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.308425 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912590b9-57ca-4f89-a071-5cd4d5757681 service glance] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.377490 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.378385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.378653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.379146 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.379535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.381437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.408943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1353/2687] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:05 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:05.411777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.425512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c659559-50db-4565-9171-8aea11f06c53 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.432722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.433555 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.433936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.434310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.435393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.437892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.450281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1335/2688] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:05.454266 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.467867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2cfb-06d6-4855-91c4-b021cc1cc837 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.474500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.475433 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.476431 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.476815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] PATH_INFO: `/v3/projects/0df9b087dafe47098e8cb5740e2c60fd/users/fe399f79f1ee47b1af497bdbcb74833f/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.495204 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df9b087dafe47098e8cb5740e2c60fd, user_id=fe399f79f1ee47b1af497bdbcb74833f, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.498577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.515211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1354/2689] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:05 2022] PUT /identity/v3/projects/0df9b087dafe47098e8cb5740e2c60fd/users/fe399f79f1ee47b1af497bdbcb74833f/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:05.520069 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.534924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a4aa2-7512-44de-854c-9495774a7a1a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.607194 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29453979-4d04-4957-9e39-16e348279d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.607885 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29453979-4d04-4957-9e39-16e348279d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.608270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29453979-4d04-4957-9e39-16e348279d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.608579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29453979-4d04-4957-9e39-16e348279d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.609332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29453979-4d04-4957-9e39-16e348279d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.611251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29453979-4d04-4957-9e39-16e348279d6e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.622546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1336/2690] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:05.626720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.639422 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6bbd6-db40-4bf1-8860-7c86c8b50801 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.645179 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.646259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.646692 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.647167 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] PATH_INFO: `/v3/projects/0df9b087dafe47098e8cb5740e2c60fd/users/fe399f79f1ee47b1af497bdbcb74833f/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.650142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df9b087dafe47098e8cb5740e2c60fd, user_id=fe399f79f1ee47b1af497bdbcb74833f, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.652974 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.666211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1355/2691] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:05 2022] PUT /identity/v3/projects/0df9b087dafe47098e8cb5740e2c60fd/users/fe399f79f1ee47b1af497bdbcb74833f/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:05.678340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.679038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.679383 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.717995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] MFA Rules not processed for user `fe399f79f1ee47b1af497bdbcb74833f`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:05.717995 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:05.805743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1337/2692] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:05.817050 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:05.818895 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:05.830280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:05.883111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e2cc6-9104-446d-aa1e-7c6a052577ab admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:05.887072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:05.887701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:05.887984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:05.888259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:05.897178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:05.899318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:05.908723 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1356/2693] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:05.990872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:06.001119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b770d2-4d92-400d-be5f-e08b5e68ba59 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:06.005423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714aea34-4c6e-4dfa-8e35-d4f7f7dc2c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:06.006018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714aea34-4c6e-4dfa-8e35-d4f7f7dc2c47 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:06.006431 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714aea34-4c6e-4dfa-8e35-d4f7f7dc2c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:06.006708 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714aea34-4c6e-4dfa-8e35-d4f7f7dc2c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:06.012276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:06.070083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714aea34-4c6e-4dfa-8e35-d4f7f7dc2c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:06.071605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714aea34-4c6e-4dfa-8e35-d4f7f7dc2c47 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:06.081023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1338/2694] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:06.929332 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:06.930801 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:06.995012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28750dc-3d00-4eac-bd54-a66cf97e409f service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:07.001396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:07.002483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:07.003000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:07.003538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] PATH_INFO: `/v3/users/3614aa5652614d9381af21acdc18fae6` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:07.005536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3614aa5652614d9381af21acdc18fae6)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:07.008828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:07.057715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3614aa5652614d9381af21acdc18fae6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.064893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3614aa5652614d9381af21acdc18fae6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.065600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3614aa5652614d9381af21acdc18fae6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.072597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3614aa5652614d9381af21acdc18fae6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.084457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3614aa5652614d9381af21acdc18fae6'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.090756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1357/2695] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:06 2022] DELETE /identity/v3/users/3614aa5652614d9381af21acdc18fae6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:07.095261 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:07.095722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:07.095818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:07.115083 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:07.116312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:07.174986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1339/2696] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:07.184771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:07.186212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:07.220977 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:07.247264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf41b4-a257-43ee-975d-e307a2f16c4a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:07.254221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:07.254850 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:07.255173 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:07.255532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:07.265931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:07.267466 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:07.277757 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1358/2697] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:07.609957 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:07.611677 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:07.628566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284f8d-4b34-4e47-a7fb-2741a614f301 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:07.633969 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:07.634618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:07.634895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:07.635197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] PATH_INFO: `/v3/projects/c189d92065934d928af13f4f22257a6d` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:07.636398 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c189d92065934d928af13f4f22257a6d)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:07.638361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:07.690359 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c189d92065934d928af13f4f22257a6d', 'request_id': 'req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.696791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c189d92065934d928af13f4f22257a6d', 'request_id': 'req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.702951 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c189d92065934d928af13f4f22257a6d', 'request_id': 'req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.716113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] The token cache is being invalidate because project c189d92065934d928af13f4f22257a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:07.716406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.717020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.718490 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1340/2698] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:07 2022] DELETE /identity/v3/projects/c189d92065934d928af13f4f22257a6d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:07.722630 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:07.732254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bba77-3236-4806-9bea-922c600f9015 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:07.792053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:07.792690 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:07.792970 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:07.793243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] PATH_INFO: `/v3/users/236200a994354560b3810f85854db8f8` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:07.794369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=236200a994354560b3810f85854db8f8)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:07.796314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:07.843115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '236200a994354560b3810f85854db8f8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.855178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '236200a994354560b3810f85854db8f8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.855870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '236200a994354560b3810f85854db8f8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.869992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '236200a994354560b3810f85854db8f8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.884217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cfb198e9-9d0b-40d3-8f64-f33c33f52c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '236200a994354560b3810f85854db8f8'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:07.891883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1359/2699] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:07 2022] DELETE /identity/v3/users/236200a994354560b3810f85854db8f8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:08.169241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.170730 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:08.184456 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c4e3-924e-4f62-afc2-6ac37d8b987c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.211386 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:08.246429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.247081 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.247405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.247835 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] PATH_INFO: `/v3/projects/b47cf6519f984f62b88fb7e9be869b7b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.249002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47cf6519f984f62b88fb7e9be869b7b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.250938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.296487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47cf6519f984f62b88fb7e9be869b7b', 'request_id': 'req-a0d90b75-7270-46d3-8798-4eecb7155273'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:08.301812 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47cf6519f984f62b88fb7e9be869b7b', 'request_id': 'req-a0d90b75-7270-46d3-8798-4eecb7155273'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:08.308645 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47cf6519f984f62b88fb7e9be869b7b', 'request_id': 'req-a0d90b75-7270-46d3-8798-4eecb7155273'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:08.321202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] The token cache is being invalidate because project b47cf6519f984f62b88fb7e9be869b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:08.321493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:08.322116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a0d90b75-7270-46d3-8798-4eecb7155273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:08.323629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1341/2700] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:08 2022] DELETE /identity/v3/projects/b47cf6519f984f62b88fb7e9be869b7b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:08.328414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.328901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.329032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.349110 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:08.350241 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:08.398651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:08.413994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1360/2701] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:08.421937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.422219 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.422494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.442379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:08.443628 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:08.451585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1342/2702] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:08.455783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.466093 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2f94a-f5df-4324-80fb-d90f95833973 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.470972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.471674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.471952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.472222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.472870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:08.473172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.474799 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.487681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1361/2703] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:39:08.492068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.501093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6f0da-6df0-4be0-981d-26440d25591d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.505920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.506520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.506796 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.507065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.507723 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.509593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.539262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1343/2704] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:08 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:08.543548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.553691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe7bc8-fa81-4080-a8de-1d467f45429b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.612726 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.613271 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.613552 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.613826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.614589 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.616753 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.637677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1362/2705] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:08 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:08.641856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.653352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40e106-3b58-4f1f-98b7-ec4c751630dc admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.658088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.658729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.659005 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.659603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.660279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.661867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.673214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1344/2706] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:08.677119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.689041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789d7f2-7463-4aca-b8f5-6a94f2fc455a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.693776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da944f-6b7f-4a2c-8e4c-bf33621be32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.694405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da944f-6b7f-4a2c-8e4c-bf33621be32e admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.694721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da944f-6b7f-4a2c-8e4c-bf33621be32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.694990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da944f-6b7f-4a2c-8e4c-bf33621be32e admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.695670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da944f-6b7f-4a2c-8e4c-bf33621be32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.697246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da944f-6b7f-4a2c-8e4c-bf33621be32e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.708449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1363/2707] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:08.713629 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.724808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080e4fb-1a09-423e-9595-44d105223042 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:08.730130 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:08.730855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.731166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.731442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] PATH_INFO: `/v3/projects/00f5eaedd71244628d5bb2814566da3e/users/add90d32a55146d48bb93cbc50189998/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.746468 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f5eaedd71244628d5bb2814566da3e, user_id=add90d32a55146d48bb93cbc50189998, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:08.749745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:08.764302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1345/2708] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:08 2022] PUT /identity/v3/projects/00f5eaedd71244628d5bb2814566da3e/users/add90d32a55146d48bb93cbc50189998/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:08.776993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:08.777400 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:08.778287 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:08.812800 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] MFA Rules not processed for user `add90d32a55146d48bb93cbc50189998`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:08.814185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:08.894697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1364/2709] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:08.940199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:08.947325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:09.004576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba62b3-161e-43d6-9c60-d396bc2374c3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:09.008747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639035fb-a7c8-4738-a422-e7243402116b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:09.009344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639035fb-a7c8-4738-a422-e7243402116b service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:09.009620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639035fb-a7c8-4738-a422-e7243402116b service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:09.009889 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639035fb-a7c8-4738-a422-e7243402116b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:09.018404 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639035fb-a7c8-4738-a422-e7243402116b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:09.019865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639035fb-a7c8-4738-a422-e7243402116b service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:09.023985 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:09.031774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1346/2710] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:18.892014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:19.035337 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:21.627305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:21.688364 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1987f5-68f7-4fed-bd1c-ba977c272612 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:21.692893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:21.693520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:21.693806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:21.694083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] PATH_INFO: `/v3/users/add90d32a55146d48bb93cbc50189998` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:21.695221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] RBAC: Authorizing `identity:delete_user(user_id=add90d32a55146d48bb93cbc50189998)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:21.697171 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:21.742035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add90d32a55146d48bb93cbc50189998'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:21.749858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add90d32a55146d48bb93cbc50189998'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:21.750541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add90d32a55146d48bb93cbc50189998'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:21.757333 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add90d32a55146d48bb93cbc50189998'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:21.768863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add90d32a55146d48bb93cbc50189998'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:21.774517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1365/2711] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:21 2022] DELETE /identity/v3/users/add90d32a55146d48bb93cbc50189998 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:21.779270 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:21.779734 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:21.779881 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:21.799263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:21.800473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:21.857562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1347/2712] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:21.867080 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:21.868558 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:21.948681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c3978-6cc0-416a-a369-65b52ecce08a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:21.952521 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:21.953201 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:21.953528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:21.953798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:21.962866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:21.964388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:21.975040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1366/2713] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:22.104308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.107423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:22.123925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679bd7e-6ac0-47fd-86c8-35974067123a None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.129147 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.129772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.131028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.131321 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.133332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:22.222142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.223618 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb57f66f-ca34-4e6a-ae8b-784bd0dd3043 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.233282 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1348/2714] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:22.339341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.340839 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:22.348721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:22.355387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81a746-c2cc-487b-ae35-21c7d68518c7 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.366990 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.367721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.368064 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.368341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] PATH_INFO: `/v3/projects/00f5eaedd71244628d5bb2814566da3e` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.370260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f5eaedd71244628d5bb2814566da3e)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.372298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.421602 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f5eaedd71244628d5bb2814566da3e', 'request_id': 'req-323662aa-4328-43cb-8a34-e39963280088'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:22.427148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f5eaedd71244628d5bb2814566da3e', 'request_id': 'req-323662aa-4328-43cb-8a34-e39963280088'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:22.433857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f5eaedd71244628d5bb2814566da3e', 'request_id': 'req-323662aa-4328-43cb-8a34-e39963280088'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:22.448493 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] The token cache is being invalidate because project 00f5eaedd71244628d5bb2814566da3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:22.448968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:22.449704 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-323662aa-4328-43cb-8a34-e39963280088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:22.451214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1367/2715] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:22 2022] DELETE /identity/v3/projects/00f5eaedd71244628d5bb2814566da3e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:22.456021 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.456399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.456675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.475808 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:22.477065 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:22.539750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1349/2716] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:22.547476 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.547763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.548040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.568432 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:22.569763 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:22.578281 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1368/2717] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:22.582608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.592326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7215e9-b481-416f-bb5c-379e60ad4d28 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.596865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.597515 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.597980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.598098 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.598873 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:22.599522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.601014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.613122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1350/2718] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:39:22.617548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.627303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cad76b-0d6b-4d99-a37c-5df875193d2d None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.631936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.632565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.632791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.633061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.633716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.636309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.669079 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1369/2719] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:22 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:22.673471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.685234 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629c9d6-4764-49e0-b576-da24893dccb0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.711904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:22.746442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.747054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.747364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.747638 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.748264 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.750190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.775416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1351/2720] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:22 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:22.776835 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.787786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0510d-c77b-4dce-becb-00007b0785fb admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.792617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.793299 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.793676 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.794029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.794688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.796358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.809884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1370/2721] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:22.811162 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.821635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5ada8-d504-4282-8517-692c00e930f3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.826928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb6b13-028f-47c8-a136-cd47ca307289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.827571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb6b13-028f-47c8-a136-cd47ca307289 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.827855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb6b13-028f-47c8-a136-cd47ca307289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.828127 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb6b13-028f-47c8-a136-cd47ca307289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.828777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb6b13-028f-47c8-a136-cd47ca307289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.830413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb6b13-028f-47c8-a136-cd47ca307289 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.841410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1352/2722] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:22.845627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:22.858584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be5b42-4140-4193-943f-74f653e310c0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:22.865971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:22.866856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.867244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.867590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] PATH_INFO: `/v3/projects/0eb4fec8393248be86fc2709f34902c3/users/21342f4928174ca5bb7d9550c173f140/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.883934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eb4fec8393248be86fc2709f34902c3, user_id=21342f4928174ca5bb7d9550c173f140, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:22.887239 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:22.900863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1371/2723] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:22 2022] PUT /identity/v3/projects/0eb4fec8393248be86fc2709f34902c3/users/21342f4928174ca5bb7d9550c173f140/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:22.913251 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76865c-f774-4fea-8103-525b365152cf None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:22.913541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76865c-f774-4fea-8103-525b365152cf None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:22.913818 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76865c-f774-4fea-8103-525b365152cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:22.941840 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1c76865c-f774-4fea-8103-525b365152cf None None] MFA Rules not processed for user `21342f4928174ca5bb7d9550c173f140`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:22.943066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1c76865c-f774-4fea-8103-525b365152cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:23.009294 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1353/2724] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:23.020534 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:23.022270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:23.108942 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfc04c-4201-4d05-a68b-a2ee4f411755 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:23.113319 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0e9de-dd62-4c89-b85d-a3645d463d07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:23.114226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e9de-dd62-4c89-b85d-a3645d463d07 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:23.114720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e9de-dd62-4c89-b85d-a3645d463d07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:23.115343 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e9de-dd62-4c89-b85d-a3645d463d07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:23.125429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e9de-dd62-4c89-b85d-a3645d463d07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:23.127012 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0e9de-dd62-4c89-b85d-a3645d463d07 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:23.146682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1372/2725] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:26.521089 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76865c-f774-4fea-8103-525b365152cf None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:26.585236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76865c-f774-4fea-8103-525b365152cf None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:26.589767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:26.590393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:26.590675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:26.590974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] PATH_INFO: `/v3/users/21342f4928174ca5bb7d9550c173f140` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:26.592184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] RBAC: Authorizing `identity:delete_user(user_id=21342f4928174ca5bb7d9550c173f140)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:26.594148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:26.632656 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:26.644151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21342f4928174ca5bb7d9550c173f140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:26.651035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21342f4928174ca5bb7d9550c173f140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:26.658135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21342f4928174ca5bb7d9550c173f140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:26.658763 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21342f4928174ca5bb7d9550c173f140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:26.671166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21342f4928174ca5bb7d9550c173f140'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:26.677163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1354/2726] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:26 2022] DELETE /identity/v3/users/21342f4928174ca5bb7d9550c173f140 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:26.681481 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:26.681957 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:26.682885 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:26.691027 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:26.703363 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:26.704597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:26.886994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1373/2727] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:26.896703 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:26.898062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:26.954659 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640273f4-6807-4e7b-be70-a04be21680be admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:26.958166 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:26.958765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:26.959038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:26.959332 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:26.967966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:26.969382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:26.978414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1355/2728] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:27.580871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:27.582208 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:27.584614 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:27.606745 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bd16e-dffe-4479-b683-3ec2c7710671 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:27.613342 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb3f2-1d3a-4e38-8713-305e2f542136 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:27.614954 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:27.616754 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:27.617274 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:27.618262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:27.621137 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:27.622592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:27.623261 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:27.623579 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:27.624021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] PATH_INFO: `/v3/projects/0eb4fec8393248be86fc2709f34902c3` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:27.625298 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eb4fec8393248be86fc2709f34902c3)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:27.627337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:27.637921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:27.639959 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ded37f-58ac-4a32-86aa-5b83e89a3ea4 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:27.660636 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1356/2729] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:27.698664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eb4fec8393248be86fc2709f34902c3', 'request_id': 'req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:27.710153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eb4fec8393248be86fc2709f34902c3', 'request_id': 'req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:27.721955 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eb4fec8393248be86fc2709f34902c3', 'request_id': 'req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:27.753913 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] The token cache is being invalidate because project 0eb4fec8393248be86fc2709f34902c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:27.754215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:27.756905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-7cddd08a-aa05-4f4e-bad6-4b354f8f2eab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:27.764273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1374/2730] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:27 2022] DELETE /identity/v3/projects/0eb4fec8393248be86fc2709f34902c3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:27.766089 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:27.766512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:27.766800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:27.793772 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:27.794972 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:27.854767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1357/2731] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:27.862562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:27.862845 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:27.863135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:27.883006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:27.884236 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:27.892911 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1375/2732] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:27.897239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:27.907248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b60df-18fe-4758-bb37-f74b3b9667fa None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:27.912115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:27.912667 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:27.912925 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:27.913193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:27.913833 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:27.914134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:27.915743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:27.927815 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1358/2733] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:39:27.932240 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:27.941259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbcb40-56d3-4f1f-833c-a1ccab17e6cf None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:27.946002 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:27.946599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:27.946874 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:27.947166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:27.947813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:27.949677 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:27.977407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1376/2734] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:27 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:27.982089 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:27.992125 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752d452-21fe-43b7-81ef-a256b632dfe5 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.021054 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:28.050934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.051662 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.051947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.052228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.052878 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.054842 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.076199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1359/2735] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:27 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:28.080377 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.091756 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81933793-f791-44a2-8ece-c99d15b75f18 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.096484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.097177 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.097453 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.097725 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.098367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.100037 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.110398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1377/2736] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:28.114437 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.124414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880789-c311-40ac-a6e3-7c2aea3b5511 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.129733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665640d-1ab1-46e0-9ccd-79308b0a959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.130353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665640d-1ab1-46e0-9ccd-79308b0a959d admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.130616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665640d-1ab1-46e0-9ccd-79308b0a959d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.130912 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665640d-1ab1-46e0-9ccd-79308b0a959d admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.131595 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665640d-1ab1-46e0-9ccd-79308b0a959d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.133206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665640d-1ab1-46e0-9ccd-79308b0a959d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.144523 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1360/2737] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:28.148755 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.159996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a5f1e-fdc1-44e3-b899-38acceb854c5 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.164595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.165184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.165464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.165738 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] PATH_INFO: `/v3/projects/7b55812622ba46a68bb5265bb4b3d390/users/701fcfb66d334256bd12447b5cc0f174/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.181805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b55812622ba46a68bb5265bb4b3d390, user_id=701fcfb66d334256bd12447b5cc0f174, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.184619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.199522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1378/2738] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:28 2022] PUT /identity/v3/projects/7b55812622ba46a68bb5265bb4b3d390/users/701fcfb66d334256bd12447b5cc0f174/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:28.209872 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.210165 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.210443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.237771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] MFA Rules not processed for user `701fcfb66d334256bd12447b5cc0f174`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:28.238992 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:28.303781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1361/2739] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:28.308168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.335930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:28.374528 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725eb6da-e6a7-471d-9806-f7322af9a9a2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.379263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.379860 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.380141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.380408 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.381021 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.382936 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.412131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1379/2740] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:28 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:28.415901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.426458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51a03-fcaa-42b6-a45b-cec159e0a60d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.483420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.484062 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.484342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.484609 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.485230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.487226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.507834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1362/2741] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:28 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:28.512246 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.522791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7b54c-4fd1-4ed0-8548-b2100288f1d4 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.528639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.529345 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.529840 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.530145 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.531036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.533215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.545285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1380/2742] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:28.549231 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.560226 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dc5d-17e9-41ca-a0c7-a6b120fb5a91 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.565455 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307cd6f-d1d8-45dc-af35-85424becace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.565579 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307cd6f-d1d8-45dc-af35-85424becace8 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.565787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307cd6f-d1d8-45dc-af35-85424becace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.566060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307cd6f-d1d8-45dc-af35-85424becace8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.566701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307cd6f-d1d8-45dc-af35-85424becace8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.568529 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307cd6f-d1d8-45dc-af35-85424becace8 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.579906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1363/2743] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:28.584252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.596211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de19bf4-1eb8-420e-85fe-50fda20d6ea2 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.603983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.605051 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.605390 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.605858 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] PATH_INFO: `/v3/projects/ff2b4b656bc2472c8fd2a32508cca9f3/users/e719c3df8a5942979d16e62a276971ca/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.626418 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2b4b656bc2472c8fd2a32508cca9f3, user_id=e719c3df8a5942979d16e62a276971ca, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.629001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.646265 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1381/2744] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:28 2022] PUT /identity/v3/projects/ff2b4b656bc2472c8fd2a32508cca9f3/users/e719c3df8a5942979d16e62a276971ca/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:28.657733 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.658193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.658308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.696235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] MFA Rules not processed for user `e719c3df8a5942979d16e62a276971ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:28.697586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:28.704784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:28.779566 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1364/2745] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:28.781165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.858759 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcabd4-ad8c-4569-963e-d3628ce9b8bc admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.865373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.865993 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.866344 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.866597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.867264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.869199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:28.911242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1382/2746] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:28 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:28.912308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:28.924688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b28571-93fc-41eb-b200-5bc03a022963 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:28.992838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:28.993538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:28.993891 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:28.994236 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:28.995520 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:28.997608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:29.024204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1365/2747] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:28 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:29.028808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:29.039883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cccceb-a81e-4283-9232-2cdf933a8bd0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:29.045212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:29.045812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:29.046085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:29.046376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:29.047223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:29.048814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:29.059635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1383/2748] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:29.064399 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:29.074161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115826e3-33fc-4079-8553-7cab16969bd4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:29.081983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:29.081983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:29.081983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:29.081983 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] PATH_INFO: `/v3/projects/b3edce452d4147559008566b7a01e027/users/cba7c87c52b44387b8411b134ae7c712/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:29.095592 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3edce452d4147559008566b7a01e027, user_id=cba7c87c52b44387b8411b134ae7c712, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:29.098512 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:29.111913 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1366/2749] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:29 2022] PUT /identity/v3/projects/b3edce452d4147559008566b7a01e027/users/cba7c87c52b44387b8411b134ae7c712/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:29.117313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:29.130036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd0796-ab80-441c-90c9-3d85b386e426 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:29.184321 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:29.200404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c32b85-4055-4544-beb6-123e29d48ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:29.201061 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c32b85-4055-4544-beb6-123e29d48ced admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:29.201537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c32b85-4055-4544-beb6-123e29d48ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:29.201906 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c32b85-4055-4544-beb6-123e29d48ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:29.202679 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c32b85-4055-4544-beb6-123e29d48ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:29.204797 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c32b85-4055-4544-beb6-123e29d48ced admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:29.217375 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1384/2750] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:29.221496 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:29.232395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b81e546-085a-449c-9df8-be6b59c4ed29 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:29.237088 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:29.237738 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:29.238059 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:29.238336 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] PATH_INFO: `/v3/projects/b3edce452d4147559008566b7a01e027/users/cba7c87c52b44387b8411b134ae7c712/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:29.240284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3edce452d4147559008566b7a01e027, user_id=cba7c87c52b44387b8411b134ae7c712, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:29.242895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:29.254361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1367/2751] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:29 2022] PUT /identity/v3/projects/b3edce452d4147559008566b7a01e027/users/cba7c87c52b44387b8411b134ae7c712/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:29.264451 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:29.264734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:29.265009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:29.294396 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] MFA Rules not processed for user `cba7c87c52b44387b8411b134ae7c712`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:29.295638 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:29.382867 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1385/2752] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:29.396028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:29.401435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:29.465809 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d4f0a-1e2b-4ee7-9e1b-a542b32df6fb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:29.471180 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:29.471967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:29.472424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:29.472695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:29.490538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:29.545493 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:29.546836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:29.558364 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1368/2753] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:32.205831 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:32.216517 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2a111-3aaa-4fce-9ff8-87412b84c1d2 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:32.220138 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:32.220737 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:32.221034 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:32.221291 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:32.280360 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:32.281802 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:32.291334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1386/2754] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:33.618545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:33.634668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13daa5-c009-4f4b-ab56-132b84f5e7fe service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:33.642549 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:33.643443 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:33.643718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:33.643993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:33.658232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:33.660138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:33.673576 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1369/2755] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:34.051844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:34.053260 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:34.121841 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f07e-7285-421b-86a6-accda6130c2a service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:34.126347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:34.126975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:34.127285 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:34.127682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] PATH_INFO: `/v3/users/c376f5bd19c840b7924093606dee6f4f` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:34.128827 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c376f5bd19c840b7924093606dee6f4f)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:34.130745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:34.141507 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:34.176684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c376f5bd19c840b7924093606dee6f4f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.184499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c376f5bd19c840b7924093606dee6f4f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.185273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c376f5bd19c840b7924093606dee6f4f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.192197 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c376f5bd19c840b7924093606dee6f4f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.205259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c376f5bd19c840b7924093606dee6f4f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.211844 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1387/2756] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:34 2022] DELETE /identity/v3/users/c376f5bd19c840b7924093606dee6f4f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:34.396341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:34.409850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceed698-d8f7-49cc-9cf4-c7a0dea21f8e service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:34.466844 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:34.467484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:34.467764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:34.468064 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] PATH_INFO: `/v3/projects/fb0c0d2823dd4a3fb35bb8c0cad4f410` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:34.469156 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb0c0d2823dd4a3fb35bb8c0cad4f410)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:34.471033 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:34.515732 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb0c0d2823dd4a3fb35bb8c0cad4f410', 'request_id': 'req-c573e789-14fd-4b66-a329-dfee4944e354'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.521131 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb0c0d2823dd4a3fb35bb8c0cad4f410', 'request_id': 'req-c573e789-14fd-4b66-a329-dfee4944e354'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.527780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb0c0d2823dd4a3fb35bb8c0cad4f410', 'request_id': 'req-c573e789-14fd-4b66-a329-dfee4944e354'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.539829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] The token cache is being invalidate because project fb0c0d2823dd4a3fb35bb8c0cad4f410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:34.540114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.540716 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.542153 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1370/2757] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:34 2022] DELETE /identity/v3/projects/fb0c0d2823dd4a3fb35bb8c0cad4f410 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:34.546168 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:34.547613 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:34.561590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c14b0e-c2e2-410c-a71b-522d12e97098 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:34.621551 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:34.622398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:34.622713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:34.623105 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] PATH_INFO: `/v3/users/02887110cc77498e983edb258d4730c0` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:34.624537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02887110cc77498e983edb258d4730c0)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:34.627126 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:34.674305 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02887110cc77498e983edb258d4730c0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.681884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02887110cc77498e983edb258d4730c0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.682734 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02887110cc77498e983edb258d4730c0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.699926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02887110cc77498e983edb258d4730c0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.714668 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02887110cc77498e983edb258d4730c0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:34.720751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1388/2758] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:34 2022] DELETE /identity/v3/users/02887110cc77498e983edb258d4730c0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:34.913247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:34.914773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:34.933193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573e789-14fd-4b66-a329-dfee4944e354 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:34.964135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:34.992838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:34.993469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:34.993744 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:34.994015 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] PATH_INFO: `/v3/projects/77d7b40b3fe54711817a1d361b10a0c2` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:34.995187 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d7b40b3fe54711817a1d361b10a0c2)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:34.997129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:35.042632 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d7b40b3fe54711817a1d361b10a0c2', 'request_id': 'req-f8f25a84-6577-427f-b258-73291b2b8b07'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:35.048314 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d7b40b3fe54711817a1d361b10a0c2', 'request_id': 'req-f8f25a84-6577-427f-b258-73291b2b8b07'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:35.054355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d7b40b3fe54711817a1d361b10a0c2', 'request_id': 'req-f8f25a84-6577-427f-b258-73291b2b8b07'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:35.066342 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] The token cache is being invalidate because project 77d7b40b3fe54711817a1d361b10a0c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:35.066634 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:35.067301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-f8f25a84-6577-427f-b258-73291b2b8b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:35.068789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1371/2759] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:34 2022] DELETE /identity/v3/projects/77d7b40b3fe54711817a1d361b10a0c2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:40.754282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:40.755933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:40.769212 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65e42-0ea0-4fe6-a3d9-de38285ea876 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:40.772741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:40.828381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:40.829006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:40.829303 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:40.829570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] PATH_INFO: `/v3/users/447e6667a9fa4ec89e53a40c0cb008ba` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:40.830683 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=447e6667a9fa4ec89e53a40c0cb008ba)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:40.832635 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:40.885318 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447e6667a9fa4ec89e53a40c0cb008ba'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:40.892828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447e6667a9fa4ec89e53a40c0cb008ba'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:40.893448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447e6667a9fa4ec89e53a40c0cb008ba'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:40.901114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447e6667a9fa4ec89e53a40c0cb008ba'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:40.914401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447e6667a9fa4ec89e53a40c0cb008ba'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:40.920257 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1389/2760] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:40 2022] DELETE /identity/v3/users/447e6667a9fa4ec89e53a40c0cb008ba => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:40.925308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:40.925830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:40.925830 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:40.944570 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:40.945768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:41.004976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1372/2761] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:41.015496 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:41.017130 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:41.074156 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62009ff-e7de-47f4-b1dd-419e3d6fecb8 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:41.077818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:41.078417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:41.078700 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:41.078975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:41.087934 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:41.090031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:41.099362 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1390/2762] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:41.367077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:41.368664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:41.381363 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba69e8-57f4-4758-90ca-62d776027a9f None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:41.386106 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:41.387463 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:41.387865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:41.388260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] PATH_INFO: `/v3/projects/ec53a2ca56b84f5dbf01c9a044ffdccd` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:41.389473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec53a2ca56b84f5dbf01c9a044ffdccd)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:41.393067 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:41.447693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec53a2ca56b84f5dbf01c9a044ffdccd', 'request_id': 'req-2f9a30f7-54a8-4af3-beda-007e234ea6b5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.453155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:41.454335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec53a2ca56b84f5dbf01c9a044ffdccd', 'request_id': 'req-2f9a30f7-54a8-4af3-beda-007e234ea6b5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.463499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec53a2ca56b84f5dbf01c9a044ffdccd', 'request_id': 'req-2f9a30f7-54a8-4af3-beda-007e234ea6b5'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.480225 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] The token cache is being invalidate because project ec53a2ca56b84f5dbf01c9a044ffdccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:41.480527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.481204 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2f9a30f7-54a8-4af3-beda-007e234ea6b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.482789 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1373/2763] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:41 2022] DELETE /identity/v3/projects/ec53a2ca56b84f5dbf01c9a044ffdccd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:39:41.796556 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:41.797951 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:41.810475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83103453-82b9-48ab-8c6b-0ee025440cd7 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:41.835644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:41.866581 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:41.867302 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:41.867539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:41.867816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] PATH_INFO: `/v3/users/701fcfb66d334256bd12447b5cc0f174` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:41.868929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=701fcfb66d334256bd12447b5cc0f174)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:41.870857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:41.919184 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701fcfb66d334256bd12447b5cc0f174'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.926572 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701fcfb66d334256bd12447b5cc0f174'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.927457 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701fcfb66d334256bd12447b5cc0f174'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.934382 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701fcfb66d334256bd12447b5cc0f174'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.945941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701fcfb66d334256bd12447b5cc0f174'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:41.952356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1391/2764] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:41 2022] DELETE /identity/v3/users/701fcfb66d334256bd12447b5cc0f174 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:41.956862 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:41.957334 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:41.957435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:41.977255 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:41.978458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:42.036243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1374/2765] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:42.045798 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:42.047311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:42.103611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33585bdf-6300-4ab6-a13d-6241141d1cc5 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:42.107882 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:42.108478 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:42.108761 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:42.109036 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:42.117883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:42.119331 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:42.128554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1392/2766] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:42.396856 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:42.398324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:42.413642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3135d-ac29-4ca4-a001-22b9cb889365 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:42.418335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:42.418963 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:42.419344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:42.419528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] PATH_INFO: `/v3/projects/7b55812622ba46a68bb5265bb4b3d390` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:42.421032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b55812622ba46a68bb5265bb4b3d390)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:42.423238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:42.469155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b55812622ba46a68bb5265bb4b3d390', 'request_id': 'req-21a702e9-a253-41b1-89ad-b95699e83eb6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.474522 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b55812622ba46a68bb5265bb4b3d390', 'request_id': 'req-21a702e9-a253-41b1-89ad-b95699e83eb6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.481037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b55812622ba46a68bb5265bb4b3d390', 'request_id': 'req-21a702e9-a253-41b1-89ad-b95699e83eb6'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.492718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] The token cache is being invalidate because project 7b55812622ba46a68bb5265bb4b3d390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:42.493009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.493627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.495074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1375/2767] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:42 2022] DELETE /identity/v3/projects/7b55812622ba46a68bb5265bb4b3d390 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:42.499674 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:42.509807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e11b-9b39-4083-8deb-148547f3b4a8 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:42.554959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:42.566956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:42.567611 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:42.567892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:42.568165 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] PATH_INFO: `/v3/users/e719c3df8a5942979d16e62a276971ca` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:42.569282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e719c3df8a5942979d16e62a276971ca)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:42.571229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:42.618017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e719c3df8a5942979d16e62a276971ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.625601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e719c3df8a5942979d16e62a276971ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.626223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e719c3df8a5942979d16e62a276971ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.633160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e719c3df8a5942979d16e62a276971ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.644687 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e719c3df8a5942979d16e62a276971ca'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:42.650369 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1393/2768] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:42 2022] DELETE /identity/v3/users/e719c3df8a5942979d16e62a276971ca => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:42.936507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:42.938267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:42.955675 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a702e9-a253-41b1-89ad-b95699e83eb6 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:43.023481 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:43.024114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:43.024393 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:43.024664 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] PATH_INFO: `/v3/projects/ff2b4b656bc2472c8fd2a32508cca9f3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:43.026370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2b4b656bc2472c8fd2a32508cca9f3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:43.028612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:43.041046 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:43.088024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2b4b656bc2472c8fd2a32508cca9f3', 'request_id': 'req-d182e65b-533f-44ec-b62a-00d70137f2ee'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.094768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2b4b656bc2472c8fd2a32508cca9f3', 'request_id': 'req-d182e65b-533f-44ec-b62a-00d70137f2ee'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.107403 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2b4b656bc2472c8fd2a32508cca9f3', 'request_id': 'req-d182e65b-533f-44ec-b62a-00d70137f2ee'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.120047 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] The token cache is being invalidate because project ff2b4b656bc2472c8fd2a32508cca9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:43.120382 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.121828 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.123624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1376/2769] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:42 2022] DELETE /identity/v3/projects/ff2b4b656bc2472c8fd2a32508cca9f3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:43.127894 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:43.129429 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:43.142007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6e4db-06e2-436e-972d-47a3a36fda84 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:43.197155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:43.197783 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:43.198062 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:43.198335 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] PATH_INFO: `/v3/users/cba7c87c52b44387b8411b134ae7c712` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:43.200031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba7c87c52b44387b8411b134ae7c712)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:43.201935 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:43.246660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba7c87c52b44387b8411b134ae7c712'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.253762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba7c87c52b44387b8411b134ae7c712'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.254378 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba7c87c52b44387b8411b134ae7c712'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.261915 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba7c87c52b44387b8411b134ae7c712'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.268922 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:43.274721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba7c87c52b44387b8411b134ae7c712'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.281848 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1394/2770] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:43 2022] DELETE /identity/v3/users/cba7c87c52b44387b8411b134ae7c712 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:43.549826 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:43.551365 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:43.564850 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e65b-533f-44ec-b62a-00d70137f2ee admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:43.626900 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:43.627620 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:43.627896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:43.628190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] PATH_INFO: `/v3/projects/b3edce452d4147559008566b7a01e027` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:43.629295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3edce452d4147559008566b7a01e027)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:43.631223 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:43.675190 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3edce452d4147559008566b7a01e027', 'request_id': 'req-0040ad73-cd7b-438e-9c33-aa577dd01bef'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.680519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3edce452d4147559008566b7a01e027', 'request_id': 'req-0040ad73-cd7b-438e-9c33-aa577dd01bef'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.686976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3edce452d4147559008566b7a01e027', 'request_id': 'req-0040ad73-cd7b-438e-9c33-aa577dd01bef'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.700038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] The token cache is being invalidate because project b3edce452d4147559008566b7a01e027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:43.700331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.700949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-0040ad73-cd7b-438e-9c33-aa577dd01bef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:43.702414 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1377/2771] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:43 2022] DELETE /identity/v3/projects/b3edce452d4147559008566b7a01e027 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:51.465078 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:51.466491 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:51.478986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00769e87-3789-472e-9706-4d83a2807974 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:51.534659 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:51.535309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:51.535599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:51.535872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] PATH_INFO: `/v3/users/bbb2a2cf1f4141928b0d4a20c9ca65b5` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:51.536981 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb2a2cf1f4141928b0d4a20c9ca65b5)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:51.538883 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:51.583971 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb2a2cf1f4141928b0d4a20c9ca65b5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:51.590958 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb2a2cf1f4141928b0d4a20c9ca65b5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:51.591599 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb2a2cf1f4141928b0d4a20c9ca65b5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:51.598519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb2a2cf1f4141928b0d4a20c9ca65b5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:51.612009 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb2a2cf1f4141928b0d4a20c9ca65b5'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:51.617825 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1395/2772] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:51 2022] DELETE /identity/v3/users/bbb2a2cf1f4141928b0d4a20c9ca65b5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:51.623114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:51.623642 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:51.623766 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:51.642163 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:51.643372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:51.666159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:51.702197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1378/2773] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:39:51.712118 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:51.713550 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:51.769703 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6242f-b601-4520-a6b7-78002b0ee94f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:51.773361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:51.773954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:51.774231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:51.774500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:51.784301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:51.785713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:51.795494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1396/2774] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:52.067052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:52.068541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:52.081020 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b671bb9-ee10-4406-bc17-6b6bf8e76b47 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:52.085788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:52.086424 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.086706 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.086980 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] PATH_INFO: `/v3/projects/2f8f5f6873e64de7a7be06a8e3577da6` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.088146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8f5f6873e64de7a7be06a8e3577da6)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:52.090572 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:52.138001 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8f5f6873e64de7a7be06a8e3577da6', 'request_id': 'req-8324f248-ae79-4c81-8fe1-f7a337dc19a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.143470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8f5f6873e64de7a7be06a8e3577da6', 'request_id': 'req-8324f248-ae79-4c81-8fe1-f7a337dc19a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.149476 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8f5f6873e64de7a7be06a8e3577da6', 'request_id': 'req-8324f248-ae79-4c81-8fe1-f7a337dc19a0'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.161605 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] The token cache is being invalidate because project 2f8f5f6873e64de7a7be06a8e3577da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:52.161895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.162513 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.164007 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1379/2775] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:52 2022] DELETE /identity/v3/projects/2f8f5f6873e64de7a7be06a8e3577da6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:52.168224 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:52.178148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ce6f-a545-4d37-b5f7-44820373a708 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:52.185702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:52.235401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:52.236040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.236324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.236596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] PATH_INFO: `/v3/users/fe399f79f1ee47b1af497bdbcb74833f` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.237710 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe399f79f1ee47b1af497bdbcb74833f)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:52.239660 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:52.284509 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe399f79f1ee47b1af497bdbcb74833f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.291522 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe399f79f1ee47b1af497bdbcb74833f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.292143 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe399f79f1ee47b1af497bdbcb74833f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.299210 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe399f79f1ee47b1af497bdbcb74833f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.311113 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-28f6b3d5-ba3c-47e0-8218-a085f42d2453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe399f79f1ee47b1af497bdbcb74833f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.317029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1397/2776] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:39:52 2022] DELETE /identity/v3/users/fe399f79f1ee47b1af497bdbcb74833f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:52.584353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:52.585938 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:52.601030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324f248-ae79-4c81-8fe1-f7a337dc19a0 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:52.661135 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:52.661884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.662111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.662387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] PATH_INFO: `/v3/projects/0df9b087dafe47098e8cb5740e2c60fd` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.663540 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df9b087dafe47098e8cb5740e2c60fd)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:52.665480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:52.711534 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df9b087dafe47098e8cb5740e2c60fd', 'request_id': 'req-fd4d86af-bd93-46cb-afb6-11c903bee0b2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.715503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:52.718060 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df9b087dafe47098e8cb5740e2c60fd', 'request_id': 'req-fd4d86af-bd93-46cb-afb6-11c903bee0b2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.726668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df9b087dafe47098e8cb5740e2c60fd', 'request_id': 'req-fd4d86af-bd93-46cb-afb6-11c903bee0b2'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.739478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] The token cache is being invalidate because project 0df9b087dafe47098e8cb5740e2c60fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:39:52.739768 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.740381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-fd4d86af-bd93-46cb-afb6-11c903bee0b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:39:52.741867 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1380/2777] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:39:52 2022] DELETE /identity/v3/projects/0df9b087dafe47098e8cb5740e2c60fd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:52.747276 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.747769 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.747886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.767361 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:52.768555 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:52.827793 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1398/2778] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:52.836714 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.837002 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.837324 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.858267 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:52.859503 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:52.869635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1381/2779] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:52.874553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:52.884264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a9dc1-4ec4-452f-930f-b90c372930fe None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:52.888871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:52.889468 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.889767 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.890040 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.890686 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:39:52.890992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:52.892622 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:52.903931 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1399/2780] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:39:52.908151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:52.918066 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7414b9-3cfc-4ce0-a181-d3c313bb640c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:52.922584 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:52.923221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:52.923499 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:52.923765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:52.924372 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:52.926253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:52.952741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1382/2781] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:52 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:39:52.956778 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:52.966682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147009b-0fe8-4925-8db4-fe4ce98a3d37 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:52.979488 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:53.025359 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.025964 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.026247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.026527 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.027185 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.029100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.050122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1400/2782] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:52 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:53.054214 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.064895 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45dc98-3c94-4cce-bb67-1d623b9b883e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.069413 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.069996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.070272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.070547 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.071209 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.072780 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.083340 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1383/2783] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:53.086946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.096862 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105deb2f-c45b-456a-93b0-fed28d425c54 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.101401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a09b0-cec3-42c5-b333-c0def645e40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.101976 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a09b0-cec3-42c5-b333-c0def645e40a admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.102256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a09b0-cec3-42c5-b333-c0def645e40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.102525 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a09b0-cec3-42c5-b333-c0def645e40a admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.103202 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a09b0-cec3-42c5-b333-c0def645e40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.104776 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a09b0-cec3-42c5-b333-c0def645e40a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.115650 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1401/2784] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:53.119548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.130148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e68d7b-adf1-4cd1-88d0-fcf316677aeb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.134764 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.135435 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.135718 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.135991 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] PATH_INFO: `/v3/projects/e676acd461734498a14f6405a4d3f318/users/ed7a2e637e9e4cd394eb4234087a3ecc/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.149936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e676acd461734498a14f6405a4d3f318, user_id=ed7a2e637e9e4cd394eb4234087a3ecc, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.152630 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.165865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1384/2785] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:53 2022] PUT /identity/v3/projects/e676acd461734498a14f6405a4d3f318/users/ed7a2e637e9e4cd394eb4234087a3ecc/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:53.176671 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.176891 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.177164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.204436 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] MFA Rules not processed for user `ed7a2e637e9e4cd394eb4234087a3ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:53.205610 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:53.268732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1402/2786] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:53.273173 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.333506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b934f-3065-42b4-ae0a-3bf959f6b21c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.338302 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.338897 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.339686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.339958 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.340581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.342471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.368199 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1385/2787] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:39:53 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:53.372242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.382684 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31d783-0555-4503-8ba6-8a4ec0c2a4d3 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.436634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.437164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.437446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.437716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.438320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.440249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.461215 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1403/2788] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:39:53 2022] POST /identity/v3/users => generated 297 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:39:53.463971 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.474767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6ba16-c62f-42fc-a141-7ca7f1929fde admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.479395 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.480627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.480908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.481178 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.481811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.483436 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.493741 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1386/2789] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:53.497596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.507341 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b5676-b073-4231-b487-88ffe549114c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.511943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.512584 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.512869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.513148 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] PATH_INFO: `/v3/projects/6e7af1a5e90949d296d7fabf413b2a68/users/c0e18bb0a913492e945c801c6f593151/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.524826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:53.528620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7af1a5e90949d296d7fabf413b2a68, user_id=c0e18bb0a913492e945c801c6f593151, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.531789 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.546477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1404/2790] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:53 2022] PUT /identity/v3/projects/6e7af1a5e90949d296d7fabf413b2a68/users/c0e18bb0a913492e945c801c6f593151/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:53.550316 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.560423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710c1ee-20a2-4c14-9b89-144089d0e278 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.607312 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:39:53.617210 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a2ea5-fad1-4ebf-9fa5-385d8ab9d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.617806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2ea5-fad1-4ebf-9fa5-385d8ab9d4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.618084 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2ea5-fad1-4ebf-9fa5-385d8ab9d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.618379 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a2ea5-fad1-4ebf-9fa5-385d8ab9d4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.619013 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2ea5-fad1-4ebf-9fa5-385d8ab9d4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.620651 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a2ea5-fad1-4ebf-9fa5-385d8ab9d4ea admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.630050 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1387/2791] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:39:53.634379 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.645092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114943c2-2223-4825-8de7-11aa90892130 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.649724 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.650365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.650646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.650926 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] PATH_INFO: `/v3/projects/6e7af1a5e90949d296d7fabf413b2a68/users/c0e18bb0a913492e945c801c6f593151/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.653450 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e7af1a5e90949d296d7fabf413b2a68, user_id=c0e18bb0a913492e945c801c6f593151, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.656122 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.666420 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1405/2792] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:39:53 2022] PUT /identity/v3/projects/6e7af1a5e90949d296d7fabf413b2a68/users/c0e18bb0a913492e945c801c6f593151/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:39:53.676693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.677196 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.677299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.704350 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] MFA Rules not processed for user `c0e18bb0a913492e945c801c6f593151`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:39:53.705471 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:53.771113 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1388/2793] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:39:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:39:53.782856 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.784864 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:39:53.841249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184f735-a777-4847-8c5f-a9407380ca06 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.844923 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.845515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.845795 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.846069 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:53.854954 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:53.857515 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:53.875924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1406/2794] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:39:53.952688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:39:53.965061 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a816d82-dc6c-4ca3-96f3-dd3450f76c01 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:39:53.968751 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8364015-d3f6-47a2-a827-4e364fe15d03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:39:53.969346 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364015-d3f6-47a2-a827-4e364fe15d03 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:39:53.969624 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364015-d3f6-47a2-a827-4e364fe15d03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:39:53.969896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8364015-d3f6-47a2-a827-4e364fe15d03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:39:54.029869 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8364015-d3f6-47a2-a827-4e364fe15d03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:39:54.031323 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8364015-d3f6-47a2-a827-4e364fe15d03 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:39:54.040389 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1389/2795] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:39:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:03.836992 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:03.899924 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f7439-376c-4f0e-a26c-2f3e07987c98 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:03.905010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:03.905645 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:03.905929 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:03.906199 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] PATH_INFO: `/v3/users/ed7a2e637e9e4cd394eb4234087a3ecc` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:03.907365 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7a2e637e9e4cd394eb4234087a3ecc)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:03.909537 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:03.956376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7a2e637e9e4cd394eb4234087a3ecc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:03.963565 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7a2e637e9e4cd394eb4234087a3ecc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:03.964172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7a2e637e9e4cd394eb4234087a3ecc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:03.971597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7a2e637e9e4cd394eb4234087a3ecc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:03.984018 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7a2e637e9e4cd394eb4234087a3ecc'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:03.986448 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:03.991595 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1407/2796] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:03 2022] DELETE /identity/v3/users/ed7a2e637e9e4cd394eb4234087a3ecc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:03.995793 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:03.996313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:03.996313 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:04.015917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:04.017043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:04.075717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1390/2797] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:04.085211 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:04.086605 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:04.142119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f2f52f-ebb4-4bbc-8e73-917f7498009c admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:04.145691 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:04.146280 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:04.146557 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:04.146826 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:04.156231 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:04.157634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:04.167060 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1408/2798] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:04.432546 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:04.434587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:04.447077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1678-3e48-464d-8412-e9c44354fc4d None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:04.452448 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:04.453087 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:04.453487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:04.453767 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] PATH_INFO: `/v3/projects/e676acd461734498a14f6405a4d3f318` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:04.454928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e676acd461734498a14f6405a4d3f318)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:04.456859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:04.465680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:04.503822 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e676acd461734498a14f6405a4d3f318', 'request_id': 'req-3df05174-16b4-4759-9565-ff5857d892f7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.509174 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e676acd461734498a14f6405a4d3f318', 'request_id': 'req-3df05174-16b4-4759-9565-ff5857d892f7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.515848 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e676acd461734498a14f6405a4d3f318', 'request_id': 'req-3df05174-16b4-4759-9565-ff5857d892f7'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.528018 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] The token cache is being invalidate because project e676acd461734498a14f6405a4d3f318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:04.528306 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.528917 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.530619 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1391/2799] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:04 2022] DELETE /identity/v3/projects/e676acd461734498a14f6405a4d3f318 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:04.534612 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:04.544181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9dd67f-352d-4cdf-9187-171afd3197d8 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:04.599504 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:04.600121 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:04.600403 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:04.600701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] PATH_INFO: `/v3/users/c0e18bb0a913492e945c801c6f593151` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:04.602446 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e18bb0a913492e945c801c6f593151)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:04.604898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:04.652984 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e18bb0a913492e945c801c6f593151'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.661859 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e18bb0a913492e945c801c6f593151'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.662492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e18bb0a913492e945c801c6f593151'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.670846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e18bb0a913492e945c801c6f593151'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.683075 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-363b54c7-f206-400f-85d8-fd4a012890cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e18bb0a913492e945c801c6f593151'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:04.689983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1409/2800] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:04 2022] DELETE /identity/v3/users/c0e18bb0a913492e945c801c6f593151 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:04.959600 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:04.961080 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:04.974581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df05174-16b4-4759-9565-ff5857d892f7 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.032649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.033563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.033800 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.034075 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] PATH_INFO: `/v3/projects/6e7af1a5e90949d296d7fabf413b2a68` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.035232 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e7af1a5e90949d296d7fabf413b2a68)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.037191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.088320 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e7af1a5e90949d296d7fabf413b2a68', 'request_id': 'req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:05.093699 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e7af1a5e90949d296d7fabf413b2a68', 'request_id': 'req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:05.099610 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e7af1a5e90949d296d7fabf413b2a68', 'request_id': 'req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:05.112122 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] The token cache is being invalidate because project 6e7af1a5e90949d296d7fabf413b2a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:05.112416 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:05.113024 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-67d48ae7-5d76-4fa4-a7bb-91d8a9cf443e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:05.114601 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1392/2801] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:04 2022] DELETE /identity/v3/projects/6e7af1a5e90949d296d7fabf413b2a68 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:05.119404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.119890 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.120001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.140007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:05.141269 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:05.158646 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:05.201853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1410/2802] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:05.209784 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.210089 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.210361 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.229984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:05.231845 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:05.239737 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1393/2803] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:05.244244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.253930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16445c3a-37c7-4c00-a6e9-877636da58c4 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.258498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.259100 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.259401 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.259675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.260320 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:40:05.260623 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.262232 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.273554 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1411/2804] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:40:05.278278 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.287816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719b0c4-33dd-4cdd-a944-44d26b6fd546 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.292284 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.292865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.293143 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.294027 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.294649 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.296562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.316142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:05.324774 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1394/2805] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:05.328497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.338815 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86cf1d-c131-46d4-85ec-e2e517ed7568 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.392989 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.393588 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.393871 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.394147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.394773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.396720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.417739 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1412/2806] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:40:05.422384 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.433275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893177f-0dd8-4bbe-9856-8e9ed652d90e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.438185 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.438773 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.439052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.439356 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.439987 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.441585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.451561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1395/2807] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:05.455309 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.465053 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbad276-db66-407b-bc56-97af1d1c088a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.470087 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83bf26f-7f72-4da0-993a-ecdb06d04f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.470669 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bf26f-7f72-4da0-993a-ecdb06d04f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.470948 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bf26f-7f72-4da0-993a-ecdb06d04f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.471263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83bf26f-7f72-4da0-993a-ecdb06d04f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.471901 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bf26f-7f72-4da0-993a-ecdb06d04f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.473477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83bf26f-7f72-4da0-993a-ecdb06d04f5c admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.484398 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1413/2808] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:05.488215 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.498240 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17726ab3-00b9-4c98-8362-8f7954851a88 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.503023 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.503696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.503978 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.504256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] PATH_INFO: `/v3/projects/b7d8a68c81b94fc99f9d5c645fd74a69/users/95f354a76f1b4503b1ac0d0df7df6984/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.518390 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d8a68c81b94fc99f9d5c645fd74a69, user_id=95f354a76f1b4503b1ac0d0df7df6984, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.521077 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.534695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1396/2809] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:05 2022] PUT /identity/v3/projects/b7d8a68c81b94fc99f9d5c645fd74a69/users/95f354a76f1b4503b1ac0d0df7df6984/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:05.545462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.545713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.545983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.572304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] MFA Rules not processed for user `95f354a76f1b4503b1ac0d0df7df6984`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:05.573498 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:05.634865 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1414/2810] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:05.639884 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.700746 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14141a6-083d-4714-a2dd-ac0af77e058a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.705341 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.705929 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.706206 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.706475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.707099 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.708993 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.734198 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1397/2811] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:05.738374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.748892 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04885e-9d2d-4a5e-a6a3-c65d6ff951a6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.760816 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:05.804843 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.805374 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.805651 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.805921 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.806530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.808458 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.829806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1415/2812] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:05 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:40:05.833726 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.844807 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6c634-ad59-4291-972a-da31ecd253f4 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.849452 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.850042 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.850326 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.850599 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.851280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.852866 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.863308 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1398/2813] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:05.867464 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.877592 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a472195-8cdd-4af7-bad3-5586b6fd64e0 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.882290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.882940 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.883264 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.883548 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] PATH_INFO: `/v3/projects/502fb49a2dca4b5db65e55fc682beb39/users/ae8cf63a427142bba0272cdd604fdf09/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.898747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=502fb49a2dca4b5db65e55fc682beb39, user_id=ae8cf63a427142bba0272cdd604fdf09, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.901417 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.915244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1416/2814] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:05 2022] PUT /identity/v3/projects/502fb49a2dca4b5db65e55fc682beb39/users/ae8cf63a427142bba0272cdd604fdf09/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:05.919299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:05.929121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d0acd-2670-4e98-82f0-4fabcd0e2bed admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:05.984432 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53b5c2-4343-4a09-a214-f7d3dc596936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:05.985031 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53b5c2-4343-4a09-a214-f7d3dc596936 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:05.985310 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53b5c2-4343-4a09-a214-f7d3dc596936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:05.985582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53b5c2-4343-4a09-a214-f7d3dc596936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:05.986213 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53b5c2-4343-4a09-a214-f7d3dc596936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:05.987846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53b5c2-4343-4a09-a214-f7d3dc596936 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:05.997695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1399/2815] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:06.001868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:06.012749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d718c-e758-4bf0-b799-bf3c78b58720 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:06.017381 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:06.018023 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.018304 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.018577 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] PATH_INFO: `/v3/projects/502fb49a2dca4b5db65e55fc682beb39/users/ae8cf63a427142bba0272cdd604fdf09/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.020566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=502fb49a2dca4b5db65e55fc682beb39, user_id=ae8cf63a427142bba0272cdd604fdf09, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:06.023228 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:06.033392 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1417/2816] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:06 2022] PUT /identity/v3/projects/502fb49a2dca4b5db65e55fc682beb39/users/ae8cf63a427142bba0272cdd604fdf09/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:06.044747 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.045245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.045344 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.072472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] MFA Rules not processed for user `ae8cf63a427142bba0272cdd604fdf09`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:06.073672 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:06.093442 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:06.141349 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1400/2817] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:06.153387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:06.155219 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:06.212029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ded3e0-c67b-48c1-a71e-65c90ffc3c25 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:06.215719 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:06.216314 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.216597 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.216870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.225616 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:06.227038 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:06.236284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1418/2818] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:06.542253 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:06.551352 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f880f-5757-4ce2-9059-22213bee6772 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:06.555911 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae8cf63a427142bba0272cdd604fdf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '502fb49a2dca4b5db65e55fc682beb39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:06.556507 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.556787 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.557058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.557696 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:40:06.558035 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:06.559691 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:06.570955 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1401/2819] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:40:06.575562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:06.584601 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12898e86-1e24-4ad4-b76d-ad537193b708 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:06.589214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae8cf63a427142bba0272cdd604fdf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '502fb49a2dca4b5db65e55fc682beb39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:06.589803 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.590085 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.590898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.591620 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:06.593499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:06.619812 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1419/2820] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:06 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:06.623857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:06.633575 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b8d3-0337-4cb8-883e-5ab3d2bfc73e tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:06.689309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae8cf63a427142bba0272cdd604fdf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '502fb49a2dca4b5db65e55fc682beb39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:06.689934 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.690216 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.690491 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.691148 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:40:06.691458 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:06.693287 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:06.705506 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1402/2821] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:40:06.740166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:06.750664 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a4ac5-4513-4859-b7ed-48c950460e51 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:06.755142 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae8cf63a427142bba0272cdd604fdf09', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '502fb49a2dca4b5db65e55fc682beb39', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:06.755751 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:06.756031 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:06.756301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] PATH_INFO: `/v3/projects/2fd338c801464d2ab8f026e7d766ca22` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:06.757406 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2fd338c801464d2ab8f026e7d766ca22)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:06.759315 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:06.803029 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:06.805863 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd338c801464d2ab8f026e7d766ca22', 'request_id': 'req-592edcb5-1361-4887-bb78-dedff4611243'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:06.812546 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd338c801464d2ab8f026e7d766ca22', 'request_id': 'req-592edcb5-1361-4887-bb78-dedff4611243'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:06.819639 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd338c801464d2ab8f026e7d766ca22', 'request_id': 'req-592edcb5-1361-4887-bb78-dedff4611243'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:06.832010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] The token cache is being invalidate because project 2fd338c801464d2ab8f026e7d766ca22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:06.832322 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:06.832983 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-592edcb5-1361-4887-bb78-dedff4611243 tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:06.834711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1420/2822] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:06 2022] DELETE /identity/v3/projects/2fd338c801464d2ab8f026e7d766ca22 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:07.021429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:07.023169 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:07.090896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0cf631-6d23-4749-9387-81d8d9a8a5cb tempest-VolumeQuotaClassesTest-1796760739 tempest-VolumeQuotaClassesTest-1796760739-project-admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:07.134191 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=100 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:07.146558 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:07.147205 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:07.147484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:07.147753 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] PATH_INFO: `/v3/users/95f354a76f1b4503b1ac0d0df7df6984` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:07.148846 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f354a76f1b4503b1ac0d0df7df6984)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:07.150750 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:07.194693 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f354a76f1b4503b1ac0d0df7df6984'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.201444 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f354a76f1b4503b1ac0d0df7df6984'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.208709 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f354a76f1b4503b1ac0d0df7df6984'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.209381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f354a76f1b4503b1ac0d0df7df6984'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.221301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f354a76f1b4503b1ac0d0df7df6984'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.227016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1403/2823] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:07 2022] DELETE /identity/v3/users/95f354a76f1b4503b1ac0d0df7df6984 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:07.232016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:07.232541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:07.232541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:07.252058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:07.253259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:07.310994 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1421/2824] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:07.320729 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:07.322136 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:07.378120 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a710a7b-89af-4231-9f9e-5026e75fb05e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:07.381680 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:07.382276 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:07.382551 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:07.382820 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:07.391916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:07.393268 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:07.402392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1404/2825] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:07.671729 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:07.673301 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:07.685930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6a88d-2b38-4660-ad6d-488a161693b6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:07.690559 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:07.691252 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:07.691535 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:07.691807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] PATH_INFO: `/v3/projects/b7d8a68c81b94fc99f9d5c645fd74a69` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:07.692956 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d8a68c81b94fc99f9d5c645fd74a69)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:07.695058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:07.741905 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d8a68c81b94fc99f9d5c645fd74a69', 'request_id': 'req-ba6f01f5-6b89-4425-bc51-d7672d72e47f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.747376 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d8a68c81b94fc99f9d5c645fd74a69', 'request_id': 'req-ba6f01f5-6b89-4425-bc51-d7672d72e47f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.753313 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d8a68c81b94fc99f9d5c645fd74a69', 'request_id': 'req-ba6f01f5-6b89-4425-bc51-d7672d72e47f'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.764810 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] The token cache is being invalidate because project b7d8a68c81b94fc99f9d5c645fd74a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:07.765111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.765786 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.767337 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1422/2826] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:07 2022] DELETE /identity/v3/projects/b7d8a68c81b94fc99f9d5c645fd74a69 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:07.771260 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:07.781221 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf379b-5f24-469d-9813-2240d2136f5f service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:07.836607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:07.837239 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:07.837521 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:07.837790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] PATH_INFO: `/v3/users/ae8cf63a427142bba0272cdd604fdf09` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:07.838916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae8cf63a427142bba0272cdd604fdf09)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:07.841473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:07.851301 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:07.887909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae8cf63a427142bba0272cdd604fdf09'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.894652 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae8cf63a427142bba0272cdd604fdf09'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.901631 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae8cf63a427142bba0272cdd604fdf09'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.902299 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae8cf63a427142bba0272cdd604fdf09'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.915331 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-18adef89-bb06-45fd-a0b8-9eadc9cf28b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae8cf63a427142bba0272cdd604fdf09'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:07.921016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1405/2827] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:07 2022] DELETE /identity/v3/users/ae8cf63a427142bba0272cdd604fdf09 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:08.180227 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.181745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:08.196819 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f01f5-6b89-4425-bc51-d7672d72e47f admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.248347 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:08.255387 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.256074 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.256518 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.256952 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] PATH_INFO: `/v3/projects/502fb49a2dca4b5db65e55fc682beb39` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.258164 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=502fb49a2dca4b5db65e55fc682beb39)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.260086 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.303290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502fb49a2dca4b5db65e55fc682beb39', 'request_id': 'req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:08.308598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502fb49a2dca4b5db65e55fc682beb39', 'request_id': 'req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:08.314887 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502fb49a2dca4b5db65e55fc682beb39', 'request_id': 'req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:08.327440 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] The token cache is being invalidate because project 502fb49a2dca4b5db65e55fc682beb39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:08.327731 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:08.328373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-e8e253a8-a1e5-4f2b-ba29-f10bb35735a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:08.329880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1423/2828] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:08 2022] DELETE /identity/v3/projects/502fb49a2dca4b5db65e55fc682beb39 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:08.334956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.335464 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.335563 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.354668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:08.355896 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:08.413857 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1406/2829] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:08.421682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.421972 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.422247 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.442155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:08.443317 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:08.451563 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1424/2830] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:08.455673 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.465247 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce5da6-58a9-42b0-a51d-4957ef2dc77c None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.469688 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.470272 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.470548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.470816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] PATH_INFO: `/v3/domains` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.471489 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80133) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:40:08.471791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.473370 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.484593 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1407/2831] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:40:08.488663 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.498217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7817a17-7d34-4096-ba8f-d8c17221368a None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.502787 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.503393 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.503672 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.503941 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.504564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.506437 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.533072 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1425/2832] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:08.537115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.547026 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a62c-0156-466e-b413-39ca258aaeff admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.601392 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.601990 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.602290 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.602574 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.603228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.605092 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.626285 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1408/2833] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:08.630484 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.640788 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac28fa-c3ea-467c-aae5-7c9dd1bbf112 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.645334 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.645950 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.646229 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.646503 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.647181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.648782 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.658765 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1426/2834] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:08.662545 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.669269 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:08.673074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3035cc9-0f06-4798-a395-35b42d7c3f0f admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.678976 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9ef62-e358-404d-af33-7b3c28c745ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.679603 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9ef62-e358-404d-af33-7b3c28c745ae admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.679908 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9ef62-e358-404d-af33-7b3c28c745ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.680144 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9ef62-e358-404d-af33-7b3c28c745ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.680781 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9ef62-e358-404d-af33-7b3c28c745ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.682362 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a9ef62-e358-404d-af33-7b3c28c745ae admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.694602 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1409/2835] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:08.698131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.708077 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef987f-cd25-4dca-abdd-156a6c6e1f4d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.713137 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.713777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.714055 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.714324 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] PATH_INFO: `/v3/projects/1a6ded4ea2ed404c8e88f20d7128e983/users/993d5e8372af4cc3855dba13f7f1f536/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.728647 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6ded4ea2ed404c8e88f20d7128e983, user_id=993d5e8372af4cc3855dba13f7f1f536, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.731866 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.744492 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1427/2836] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:08 2022] PUT /identity/v3/projects/1a6ded4ea2ed404c8e88f20d7128e983/users/993d5e8372af4cc3855dba13f7f1f536/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:08.755309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.755640 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.755922 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.782616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] MFA Rules not processed for user `993d5e8372af4cc3855dba13f7f1f536`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:08.784112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:08.849375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1410/2837] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:08.853796 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.915103 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a9f9f-bd71-4533-b130-00c70628506d admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:08.920256 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:08.920857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:08.921139 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:08.921414 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:08.922032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:08.923973 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:08.949807 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1428/2838] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:08.953921 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:08.964653 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304744-9ed8-49bf-a882-5a27fbff22dd None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:09.020076 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:09.020666 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.020947 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.021241 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] PATH_INFO: `/v3/users` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.021863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:09.023813 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:09.044608 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1411/2839] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:08 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:09.048575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:09.058732 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91270c4-f914-4ca9-b25c-51d5dddafd3b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:09.063358 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:09.063946 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.064226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.064497 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.065135 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:09.066722 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:09.068830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:09.078693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1429/2840] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:09.082243 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:09.092472 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ec8a2-c9f4-4b40-b80a-ecace7bf4e3a admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:09.096974 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:09.097611 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.097887 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.098159 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] PATH_INFO: `/v3/projects/4ad0c67aa365478e8942224c554b1f8a/users/a51b6466052e4587a7a0f7c07ea3a4f0/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.113045 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad0c67aa365478e8942224c554b1f8a, user_id=a51b6466052e4587a7a0f7c07ea3a4f0, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:09.115758 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:09.129279 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1412/2841] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:09 2022] PUT /identity/v3/projects/4ad0c67aa365478e8942224c554b1f8a/users/a51b6466052e4587a7a0f7c07ea3a4f0/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:09.133181 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:09.143346 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f3577d-0f8d-41c6-97b8-c91c557e1c67 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:09.198790 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7eb79-92cb-4d68-80ce-9e47aeaba736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:09.199407 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7eb79-92cb-4d68-80ce-9e47aeaba736 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.199681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7eb79-92cb-4d68-80ce-9e47aeaba736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.200003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7eb79-92cb-4d68-80ce-9e47aeaba736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.200575 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7eb79-92cb-4d68-80ce-9e47aeaba736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:09.202144 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7eb79-92cb-4d68-80ce-9e47aeaba736 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:09.211499 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1430/2842] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:09.216068 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:09.226186 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f1dd5-c6c2-4a52-a162-e8e645fb9606 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:09.231353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:09.231996 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.232275 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.232544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] PATH_INFO: `/v3/projects/4ad0c67aa365478e8942224c554b1f8a/users/a51b6466052e4587a7a0f7c07ea3a4f0/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.234474 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad0c67aa365478e8942224c554b1f8a, user_id=a51b6466052e4587a7a0f7c07ea3a4f0, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:09.237212 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:09.247248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1413/2843] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:09 2022] PUT /identity/v3/projects/4ad0c67aa365478e8942224c554b1f8a/users/a51b6466052e4587a7a0f7c07ea3a4f0/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:40:09.313014 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.313598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.313598 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.341707 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] MFA Rules not processed for user `a51b6466052e4587a7a0f7c07ea3a4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:09.342920 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:09.409899 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1431/2844] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:09.420548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:09.422248 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:09.462586 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:09.481663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a72cb-c91b-4032-a588-e8676037464d admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:09.485325 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b9301-d38f-4dff-8279-bf382af73a27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:09.485924 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9301-d38f-4dff-8279-bf382af73a27 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:09.486208 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9301-d38f-4dff-8279-bf382af73a27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:09.486484 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b9301-d38f-4dff-8279-bf382af73a27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:09.495391 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9301-d38f-4dff-8279-bf382af73a27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:09.496823 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b9301-d38f-4dff-8279-bf382af73a27 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:09.506117 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1414/2845] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:10.035627 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:10.096115 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3226c-45e6-4c19-b0d2-0b3a19a3c072 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:10.100701 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:10.101316 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:10.101590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:10.101884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] PATH_INFO: `/v3/users/993d5e8372af4cc3855dba13f7f1f536` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:10.103003 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=993d5e8372af4cc3855dba13f7f1f536)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:10.104930 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:10.126619 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:10.153172 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993d5e8372af4cc3855dba13f7f1f536'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.160405 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993d5e8372af4cc3855dba13f7f1f536'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.161133 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993d5e8372af4cc3855dba13f7f1f536'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.167968 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993d5e8372af4cc3855dba13f7f1f536'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.180010 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993d5e8372af4cc3855dba13f7f1f536'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.186274 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1432/2846] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:10 2022] DELETE /identity/v3/users/993d5e8372af4cc3855dba13f7f1f536 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:10.190821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:10.191126 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:10.191406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:10.209509 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:10.210704 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:10.267594 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1415/2847] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 14:40:10.277134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:10.278688 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:10.334242 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb087a9-97a3-4c50-b382-90e8dc9ab99a admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:10.337886 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:10.338473 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:10.338748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:10.339017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:10.347779 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:10.349178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:10.358832 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1433/2848] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:10.621859 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:10.623311 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:10.636494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4be10-9519-46c6-aa9a-79cd4d590204 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:10.641354 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:10.642079 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:10.642446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:10.642757 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] PATH_INFO: `/v3/projects/1a6ded4ea2ed404c8e88f20d7128e983` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:10.643949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6ded4ea2ed404c8e88f20d7128e983)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:10.645901 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:10.692616 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6ded4ea2ed404c8e88f20d7128e983', 'request_id': 'req-6917c9aa-13fa-48a9-a6af-f6066c7c639c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.698262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6ded4ea2ed404c8e88f20d7128e983', 'request_id': 'req-6917c9aa-13fa-48a9-a6af-f6066c7c639c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.704571 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6ded4ea2ed404c8e88f20d7128e983', 'request_id': 'req-6917c9aa-13fa-48a9-a6af-f6066c7c639c'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.717202 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] The token cache is being invalidate because project 1a6ded4ea2ed404c8e88f20d7128e983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:10.717494 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.718112 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.719627 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1416/2849] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:10 2022] DELETE /identity/v3/projects/1a6ded4ea2ed404c8e88f20d7128e983 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:10.723562 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:10.733773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2605c6-a109-4eeb-ac41-73862a726d20 service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:10.788233 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:10.788853 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:10.789131 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:10.789404 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] PATH_INFO: `/v3/users/a51b6466052e4587a7a0f7c07ea3a4f0` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:10.790505 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51b6466052e4587a7a0f7c07ea3a4f0)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:10.792462 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:10.835221 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:10.837748 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51b6466052e4587a7a0f7c07ea3a4f0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.846791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51b6466052e4587a7a0f7c07ea3a4f0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.847633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51b6466052e4587a7a0f7c07ea3a4f0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.854721 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51b6466052e4587a7a0f7c07ea3a4f0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.866643 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-cb5cb609-ccab-440e-963e-f2f739c38ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51b6466052e4587a7a0f7c07ea3a4f0'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:10.872470 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1434/2850] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:10 2022] DELETE /identity/v3/users/a51b6466052e4587a7a0f7c07ea3a4f0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:11.144984 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.146776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:11.162304 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6917c9aa-13fa-48a9-a6af-f6066c7c639c admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.219355 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.219986 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.220263 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.220533 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] PATH_INFO: `/v3/projects/4ad0c67aa365478e8942224c554b1f8a` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.221695 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad0c67aa365478e8942224c554b1f8a)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.223697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.225906 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:11.270009 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad0c67aa365478e8942224c554b1f8a', 'request_id': 'req-59c544d5-117d-4793-b11b-d828af976d12'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:11.275771 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad0c67aa365478e8942224c554b1f8a', 'request_id': 'req-59c544d5-117d-4793-b11b-d828af976d12'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:11.281919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad0c67aa365478e8942224c554b1f8a', 'request_id': 'req-59c544d5-117d-4793-b11b-d828af976d12'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:11.294683 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] The token cache is being invalidate because project 4ad0c67aa365478e8942224c554b1f8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:11.294960 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:11.295607 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-59c544d5-117d-4793-b11b-d828af976d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:11.297086 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1417/2851] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:11 2022] DELETE /identity/v3/projects/4ad0c67aa365478e8942224c554b1f8a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:11.301873 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.302159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.302434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.321657 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:11.322855 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:11.380520 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1435/2852] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:11.388582 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.388868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.389138 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.408759 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:11.409949 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:11.417837 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1418/2853] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:11.421943 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.431654 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ef809-da05-43a4-9af5-4afc04ebd1a6 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.436762 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.437348 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.437625 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.437893 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.438530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:40:11.438880 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.440502 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.452275 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1436/2854] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 14:40:11.456256 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.465401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a524d-bdb1-4707-adc6-47d3f1633924 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.470036 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.470647 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.470927 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.471235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.471865 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.473777 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.500155 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1419/2855] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:11 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:11.504119 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.514600 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59246787-c5d3-4e92-9be7-8b5058011141 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.568590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.569186 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.569469 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.569741 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.570366 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.572311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.592714 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1437/2856] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:11 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:11.596892 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.608146 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8ebf9-11e2-4e43-8c7d-c571013859a3 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.612717 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.613305 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.613581 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.613849 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.614485 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.616116 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.626377 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1420/2857] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:11.630153 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.639914 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52271c25-8b67-447f-a5e4-f33b53b8399e admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.644409 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bbdbbf-2b86-4c9a-920a-a2cacf1d42ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.644986 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbdbbf-2b86-4c9a-920a-a2cacf1d42ac admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.645263 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbdbbf-2b86-4c9a-920a-a2cacf1d42ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.645532 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bbdbbf-2b86-4c9a-920a-a2cacf1d42ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.646166 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbdbbf-2b86-4c9a-920a-a2cacf1d42ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.647777 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bbdbbf-2b86-4c9a-920a-a2cacf1d42ac admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.658311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1438/2858] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:11.662262 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.672541 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460fd44-1a97-43e5-8bab-1af3335157b2 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.677245 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.677903 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.678184 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.678457 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] PATH_INFO: `/v3/projects/af593237b08247b3a0fff2c5b144ec2f/users/ad196baf4979435a9184b0264dfe5299/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.693502 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af593237b08247b3a0fff2c5b144ec2f, user_id=ad196baf4979435a9184b0264dfe5299, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.696228 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.708333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1421/2859] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:11 2022] PUT /identity/v3/projects/af593237b08247b3a0fff2c5b144ec2f/users/ad196baf4979435a9184b0264dfe5299/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:11.713254 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371556-8b0c-4302-8781-7910379407c0 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.713539 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371556-8b0c-4302-8781-7910379407c0 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.713830 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e371556-8b0c-4302-8781-7910379407c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.733869 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-1e371556-8b0c-4302-8781-7910379407c0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:11.735058 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-1e371556-8b0c-4302-8781-7910379407c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:11.755160 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:11.794311 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1439/2860] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:11.803855 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:11.805569 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:11.862838 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3a9e-aa0e-4584-a177-196431e9f709 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:11.866510 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:11.867142 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:11.867429 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:11.867701 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:11.877330 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:11.878776 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:11.888230 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1422/2861] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 14:40:12.484898 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.485395 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.485519 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.514017 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] MFA Rules not processed for user `ad196baf4979435a9184b0264dfe5299`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:12.515467 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:12.581350 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1440/2862] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:12.585967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:12.596467 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9eb18-f014-42de-b984-4e3f45f22a30 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:12.600952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:12.601556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.601829 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.602104 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.602791 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:12.604839 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:12.632953 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1423/2863] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:12 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 14:40:12.637681 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:12.648516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f876a-8c1f-4d04-8bae-47a7ef9bb400 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:12.705117 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:12.705720 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.706000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.706270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.706903 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:12.708828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:12.729150 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1441/2864] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:12 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:12.733292 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:12.743899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3b8f8-4f01-4eb6-8f5e-6b566cc5e4ab admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:12.748994 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:12.749585 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.749863 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.750134 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.750770 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:12.752394 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:12.754409 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:12.765016 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1424/2865] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:12.769006 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:12.778747 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff33bf-4da7-4fdf-8e3b-95a4aae47f67 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:12.783564 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:12.784200 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.784477 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.784745 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] PATH_INFO: `/v3/projects/7342d48e8afa4ad7bf422dba085a58b6/users/5cd7f3c6c33b4a28b2b1d64ba8cd4a98/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.799373 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7342d48e8afa4ad7bf422dba085a58b6, user_id=5cd7f3c6c33b4a28b2b1d64ba8cd4a98, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:12.802007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:12.815749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1442/2866] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:12 2022] PUT /identity/v3/projects/7342d48e8afa4ad7bf422dba085a58b6/users/5cd7f3c6c33b4a28b2b1d64ba8cd4a98/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:12.819705 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:12.830111 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e71048-3fb3-4667-a64b-67de38d0523e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:12.887665 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88520b4-2602-42c5-9967-0093b48d744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:12.888281 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88520b4-2602-42c5-9967-0093b48d744e admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.888561 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88520b4-2602-42c5-9967-0093b48d744e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.888834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88520b4-2602-42c5-9967-0093b48d744e admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.889470 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88520b4-2602-42c5-9967-0093b48d744e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:12.891119 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88520b4-2602-42c5-9967-0093b48d744e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:12.900682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1425/2867] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:12.904870 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:12.915083 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916872e-5ad2-4b0b-99d4-b89e8a24d74b admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:12.919596 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:12.920225 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:12.920500 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:12.920770 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] PATH_INFO: `/v3/projects/7342d48e8afa4ad7bf422dba085a58b6/users/5cd7f3c6c33b4a28b2b1d64ba8cd4a98/roles/2708400faa2c49aaa3f8bf315b82dd8b` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:12.922644 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7342d48e8afa4ad7bf422dba085a58b6, user_id=5cd7f3c6c33b4a28b2b1d64ba8cd4a98, role_id=2708400faa2c49aaa3f8bf315b82dd8b)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:12.925273 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:12.935713 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1443/2868] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:12 2022] PUT /identity/v3/projects/7342d48e8afa4ad7bf422dba085a58b6/users/5cd7f3c6c33b4a28b2b1d64ba8cd4a98/roles/2708400faa2c49aaa3f8bf315b82dd8b => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:13.555487 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:13.555778 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:13.556056 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:13.584648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] MFA Rules not processed for user `5cd7f3c6c33b4a28b2b1d64ba8cd4a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:13.585527 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:13.655928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1426/2869] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:13.667454 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:13.669773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:13.680516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:13.734996 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf683-6b0a-4aec-a549-314c77b5a754 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:13.738617 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:13.739270 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:13.739553 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:13.739829 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:13.795249 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:13.796685 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:13.805424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1444/2870] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:15.324152 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:15.326058 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:15.337690 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c095006-3df8-4e6b-8677-3a986b464fd8 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:15.341782 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:15.342366 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:15.342644 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:15.342916 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:15.351697 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:15.353043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:15.362402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1427/2871] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:23.801501 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=44 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:25.358375 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=70 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:28.225217 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:28.288570 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d359c-f642-4f79-a360-c4a392f887b4 service cinder] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:28.292988 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:28.293603 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:28.293879 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:28.294151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] PATH_INFO: `/v3/users/ad196baf4979435a9184b0264dfe5299` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:28.295804 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad196baf4979435a9184b0264dfe5299)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:28.297695 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:28.341884 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad196baf4979435a9184b0264dfe5299'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.348791 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad196baf4979435a9184b0264dfe5299'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.349385 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad196baf4979435a9184b0264dfe5299'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.356389 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad196baf4979435a9184b0264dfe5299'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.368506 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad196baf4979435a9184b0264dfe5299'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.374876 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1445/2872] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:28 2022] DELETE /identity/v3/users/ad196baf4979435a9184b0264dfe5299 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:28.548967 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:28.561129 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4da759-0340-421f-915a-30e6b62eb715 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:28.618715 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:28.619378 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:28.619657 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:28.619926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] PATH_INFO: `/v3/projects/af593237b08247b3a0fff2c5b144ec2f` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:28.621014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=af593237b08247b3a0fff2c5b144ec2f)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:28.622904 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:28.667821 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af593237b08247b3a0fff2c5b144ec2f', 'request_id': 'req-e727870d-9087-4d47-924f-cfe93a8d39cb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.673883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af593237b08247b3a0fff2c5b144ec2f', 'request_id': 'req-e727870d-9087-4d47-924f-cfe93a8d39cb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.679868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af593237b08247b3a0fff2c5b144ec2f', 'request_id': 'req-e727870d-9087-4d47-924f-cfe93a8d39cb'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.691966 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] The token cache is being invalidate because project af593237b08247b3a0fff2c5b144ec2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:28.692259 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.692883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.694406 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1428/2873] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:28 2022] DELETE /identity/v3/projects/af593237b08247b3a0fff2c5b144ec2f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:28.698460 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:28.699933 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:28.713035 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03025be0-b642-4f01-b8f9-f4723f33f396 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:28.768159 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:28.768702 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:28.768978 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:28.769248 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] PATH_INFO: `/v3/users/5cd7f3c6c33b4a28b2b1d64ba8cd4a98` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:28.770353 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cd7f3c6c33b4a28b2b1d64ba8cd4a98)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:28.772284 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:28.794925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:28.821340 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cd7f3c6c33b4a28b2b1d64ba8cd4a98'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.828805 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cd7f3c6c33b4a28b2b1d64ba8cd4a98'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.829423 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cd7f3c6c33b4a28b2b1d64ba8cd4a98'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.836216 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cd7f3c6c33b4a28b2b1d64ba8cd4a98'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.848475 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-ef3527b3-9e4b-463f-9c9e-5858c7ae54d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cd7f3c6c33b4a28b2b1d64ba8cd4a98'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:28.854151 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1446/2874] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:28 2022] DELETE /identity/v3/users/5cd7f3c6c33b4a28b2b1d64ba8cd4a98 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:40:29.022556 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.024078 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:29.038022 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727870d-9087-4d47-924f-cfe93a8d39cb admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.094686 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.095353 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.095635 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.095909 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] PATH_INFO: `/v3/projects/7342d48e8afa4ad7bf422dba085a58b6` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.097028 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7342d48e8afa4ad7bf422dba085a58b6)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.098928 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.132121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:29.146151 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7342d48e8afa4ad7bf422dba085a58b6', 'request_id': 'req-49558f25-69d9-4f8b-ab58-279e3700518a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:29.151410 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7342d48e8afa4ad7bf422dba085a58b6', 'request_id': 'req-49558f25-69d9-4f8b-ab58-279e3700518a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:29.157669 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7342d48e8afa4ad7bf422dba085a58b6', 'request_id': 'req-49558f25-69d9-4f8b-ab58-279e3700518a'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:29.170591 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] The token cache is being invalidate because project 7342d48e8afa4ad7bf422dba085a58b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80133) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:29.170880 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:29.171538 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-49558f25-69d9-4f8b-ab58-279e3700518a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:29.173291 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1429/2875] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:29 2022] DELETE /identity/v3/projects/7342d48e8afa4ad7bf422dba085a58b6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:29.178195 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.178483 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.178773 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.198294 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:29.199516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:29.256730 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1447/2876] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:29.265008 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.265288 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.265562 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.285271 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.auth.core [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80133) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:29.286387 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:29.294229 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1430/2877] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:29.298516 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.308813 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5c0b2-8a52-4efb-a54c-5ec273f99d03 None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.314046 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.314655 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.314938 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.315244 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] PATH_INFO: `/v3/domains` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.315904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80134) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 14:40:29.316209 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.317868 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.330851 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1448/2878] 10.222.0.50 () {58 vars in 1153 bytes} [Mon Aug 8 14:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 14:40:29.335557 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.344979 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9c98b-90e7-4966-b430-124fb5a1cea0 None None] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.349868 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.350475 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.350811 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.351114 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.351743 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.353682 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.383735 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1431/2879] 10.222.0.50 () {60 vars in 1153 bytes} [Mon Aug 8 14:40:29 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:29.387744 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.398258 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9c12f-1742-43c5-823d-f27cf1be4130 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.454223 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.454828 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.455134 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.455410 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.456041 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.457975 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.478004 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1449/2880] 10.222.0.50 () {60 vars in 1144 bytes} [Mon Aug 8 14:40:29 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 14:40:29.482115 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.492871 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700be5f5-eda0-4903-b128-66e3a2100a2e admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.497516 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.498109 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.498937 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.499280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] PATH_INFO: `/v3/roles` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.499926 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.501508 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.511295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1432/2881] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:29.515434 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.525367 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d5486-f3c3-47e2-9f1f-bce9fdce85f1 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.530000 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8783ec-1fcc-41b2-a91b-67aa71d70806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.530872 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8783ec-1fcc-41b2-a91b-67aa71d70806 admin admin] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.531222 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8783ec-1fcc-41b2-a91b-67aa71d70806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.531538 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8783ec-1fcc-41b2-a91b-67aa71d70806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.532192 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8783ec-1fcc-41b2-a91b-67aa71d70806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.534569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8783ec-1fcc-41b2-a91b-67aa71d70806 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.545925 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1450/2882] 10.222.0.50 () {58 vars in 1122 bytes} [Mon Aug 8 14:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 14:40:29.549952 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.560335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88737cd-d74c-4963-99e0-b95e733c2714 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.565550 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.566193 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.566478 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.566760 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] PATH_INFO: `/v3/projects/06ab2878134047d0946940638f85ee54/users/76e6c286374247c6beb084cf474504ed/roles/575750f3d1c5499bbbb22af90adfb176` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.580956 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ab2878134047d0946940638f85ee54, user_id=76e6c286374247c6beb084cf474504ed, role_id=575750f3d1c5499bbbb22af90adfb176)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.583668 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.596548 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1433/2883] 10.222.0.50 () {60 vars in 1483 bytes} [Mon Aug 8 14:40:29 2022] PUT /identity/v3/projects/06ab2878134047d0946940638f85ee54/users/76e6c286374247c6beb084cf474504ed/roles/575750f3d1c5499bbbb22af90adfb176 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 14:40:29.609132 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.609419 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.609697 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.617715 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:29.638959 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] MFA Rules not processed for user `76e6c286374247c6beb084cf474504ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:29.640214 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:29.703583 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1451/2884] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:29.714488 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:29.716197 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:29.773371 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceae76-aef5-4d9d-9c3c-92caaaeeb6fe admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:29.777010 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd22e2d936518407fba2d0be379598ce6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:29.777604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:29.777883 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:29.778157 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:29.786617 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:29.788074 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:29.797722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1434/2885] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:39.700673 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=30 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:39.793014 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:46.758633 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:46.760147 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:46.816001 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd66e5-6bf9-46e2-bf2d-1cbfa7bbc0cd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:46.819658 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:46.820259 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:46.820536 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:46.820808 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:46.822618 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:46.914949 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:46.916411 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd99da-6bd4-4181-b622-861b6536973a service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:46.936530 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1452/2886] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:47.302423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:47.364052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581cb76-4496-47e7-be7a-e1c4cb8eb4c4 service cinder] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:47.368544 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:47.369164 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:47.369438 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:47.369707 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] PATH_INFO: `/v3/users/76e6c286374247c6beb084cf474504ed` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:47.370806 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e6c286374247c6beb084cf474504ed)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:47.373238 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:47.417731 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e6c286374247c6beb084cf474504ed'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.425888 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e6c286374247c6beb084cf474504ed'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.432882 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e6c286374247c6beb084cf474504ed'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.433604 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e6c286374247c6beb084cf474504ed'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.445899 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e6c286374247c6beb084cf474504ed'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.452328 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1435/2887] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:40:47 2022] DELETE /identity/v3/users/76e6c286374247c6beb084cf474504ed => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:47.456727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] REQUEST_METHOD: `POST` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:47.457016 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:47.457327 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:47.475567 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.auth.core [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] MFA Rules not processed for user `a4f3ff67989b433e88e7bd1a0058bb7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80134) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 14:40:47.476904 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:47.533255 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1453/2888] 10.222.0.50 () {58 vars in 958 bytes} [Mon Aug 8 14:40:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 14:40:47.542722 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:47.544139 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:47.599836 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023705e4-0fad-4941-b80f-4837b056d190 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:47.603621 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:47.604093 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:47.604450 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:47.604587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:47.615405 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:47.616834 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:47.626790 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1436/2889] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:47.908566 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:47.910049 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:47.924682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d5e62-e3fb-4a45-a1b4-3c1be1522fdd None None] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:47.929452 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:47.930090 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:47.930356 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:47.930634 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] PATH_INFO: `/v3/projects/06ab2878134047d0946940638f85ee54` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:47.931771 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ab2878134047d0946940638f85ee54)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:47.933682 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:47.945114 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=100 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:47.982032 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ab2878134047d0946940638f85ee54', 'request_id': 'req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.987494 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ab2878134047d0946940638f85ee54', 'request_id': 'req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:47.993449 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ab2878134047d0946940638f85ee54', 'request_id': 'req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:48.006766 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] The token cache is being invalidate because project 06ab2878134047d0946940638f85ee54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:40:48.007056 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:48.007699 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:40:48.009162 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1454/2890] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:40:47 2022] DELETE /identity/v3/projects/06ab2878134047d0946940638f85ee54 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:40:48.113519 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:40:48.114920 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:48.127473 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388c65c-4888-4710-88b1-fd2c34c42f84 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:40:48.148785 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:48.186295 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:40:48.186919 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:40:48.187235 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:40:48.187511 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:40:48.189441 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:40:48.270936 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:40:48.272430 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:40:48.284480 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1437/2891] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:40:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:40:58.003590 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:40:58.279587 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=44 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:41:00.600693 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:41:00.602235 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:41:00.615011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92596ec3-1e9d-4f1e-8e4e-bb75640c5f37 admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:41:00.618541 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '511ecf1b54294d088d4737c1841cad5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:41:00.619155 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] REQUEST_METHOD: `GET` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:41:00.619439 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:41:00.619711 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:41:00.621388 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:41:00.684857 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:41:00.686282 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:41:00.695902 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1455/2892] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:41:06.607167 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:41:06.608727 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:41:06.624335 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc88c73-a66b-4bf0-beaa-d6173ea34696 service neutron] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:41:06.629189 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:41:06.629816 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:41:06.630121 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:41:06.630402 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] PATH_INFO: `/v3/users/2495d3c63dc44049872afb908ca6133b` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:41:06.631524 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2495d3c63dc44049872afb908ca6133b)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:41:06.633446 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:41:06.696401 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2495d3c63dc44049872afb908ca6133b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:06.705037 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2495d3c63dc44049872afb908ca6133b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:06.714043 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2495d3c63dc44049872afb908ca6133b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:06.714765 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2495d3c63dc44049872afb908ca6133b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:06.730030 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2495d3c63dc44049872afb908ca6133b'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:06.738981 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1438/2893] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:41:06 2022] DELETE /identity/v3/users/2495d3c63dc44049872afb908ca6133b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:41:06.921985 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:41:06.932614 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdffdcdc-7f56-4276-a4ba-ec6775a9654d service neutron] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:41:06.990007 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:41:06.990569 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:41:06.990846 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:41:06.991141 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] PATH_INFO: `/v3/projects/8aac3d3b720c45e8817bda1fe4d2d157` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:41:06.992243 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aac3d3b720c45e8817bda1fe4d2d157)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:41:06.994111 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:41:07.039814 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aac3d3b720c45e8817bda1fe4d2d157', 'request_id': 'req-6cbb4a11-e59a-4674-922f-0ab135ccaaae'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.045733 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aac3d3b720c45e8817bda1fe4d2d157', 'request_id': 'req-6cbb4a11-e59a-4674-922f-0ab135ccaaae'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.051629 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aac3d3b720c45e8817bda1fe4d2d157', 'request_id': 'req-6cbb4a11-e59a-4674-922f-0ab135ccaaae'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.063806 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] The token cache is being invalidate because project 8aac3d3b720c45e8817bda1fe4d2d157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:41:07.064092 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.064727 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.066178 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1456/2894] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:41:06 2022] DELETE /identity/v3/projects/8aac3d3b720c45e8817bda1fe4d2d157 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:41:07.070207 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:41:07.071633 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:41:07.085004 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fec193-f7bd-432d-9883-71ba255105ab admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:41:07.140381 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:41:07.141000 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:41:07.141280 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:41:07.141552 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] PATH_INFO: `/v3/users/16e72667b9ff4e86858c9fd057b47de3` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:41:07.142663 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16e72667b9ff4e86858c9fd057b47de3)` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:41:07.144612 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:41:07.189590 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16e72667b9ff4e86858c9fd057b47de3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.196528 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16e72667b9ff4e86858c9fd057b47de3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.203176 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16e72667b9ff4e86858c9fd057b47de3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.203788 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16e72667b9ff4e86858c9fd057b47de3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.211420 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:41:07.216469 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.notifications [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16e72667b9ff4e86858c9fd057b47de3'} {{(pid=80133) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.223333 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1439/2895] 10.222.0.50 () {58 vars in 1224 bytes} [Mon Aug 8 14:41:07 2022] DELETE /identity/v3/users/16e72667b9ff4e86858c9fd057b47de3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:41:07.395226 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Authenticating user token {{(pid=80134) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:41:07.396675 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.fernet_utils [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80134) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:41:07.411749 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb4a11-e59a-4674-922f-0ab135ccaaae admin admin] Validating token access rules against request {{(pid=80134) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:41:07.437068 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:41:07.475391 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4f3ff67989b433e88e7bd1a0058bb7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '20988f0d51544772b78672526cf109cb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80134) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:41:07.476011 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:41:07.476290 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] SCRIPT_NAME: `/identity` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:41:07.476561 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] PATH_INFO: `/v3/projects/c21408fcc6b64efa8d742f6bdf415ef9` {{(pid=80134) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:41:07.477670 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21408fcc6b64efa8d742f6bdf415ef9)` {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:41:07.479587 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] RBAC: Authorization granted {{(pid=80134) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:41:07.524253 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21408fcc6b64efa8d742f6bdf415ef9', 'request_id': 'req-86e25462-e104-4ffd-ac04-68caee46989d'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.529653 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21408fcc6b64efa8d742f6bdf415ef9', 'request_id': 'req-86e25462-e104-4ffd-ac04-68caee46989d'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.535818 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21408fcc6b64efa8d742f6bdf415ef9', 'request_id': 'req-86e25462-e104-4ffd-ac04-68caee46989d'} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.547424 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] The token cache is being invalidate because project c21408fcc6b64efa8d742f6bdf415ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80134) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 14:41:07.547716 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.548338 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG keystone.notifications [None req-86e25462-e104-4ffd-ac04-68caee46989d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80134) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 14:41:07.549792 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: [pid: 80134|app: 0|req: 1457/2896] 10.222.0.50 () {58 vars in 1233 bytes} [Mon Aug 8 14:41:07 2022] DELETE /identity/v3/projects/c21408fcc6b64efa8d742f6bdf415ef9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 14:41:17.220583 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:41:17.544574 dv-cinder-852009-2-iscsi devstack@keystone.service[80134]: DEBUG dbcounter [-] [80134] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80134) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 14:44:07.691266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Authenticating user token {{(pid=80133) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 14:44:07.692798 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:44:07.760038 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a696b17f-0b7d-4340-b251-f6cfdc503966 admin admin] Validating token access rules against request {{(pid=80133) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 14:44:07.817423 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '90fd9b3148004cc2b8d14bced329b10b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad4e032807bf4ffb9242bf1c4e011a9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80133) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 14:44:07.818032 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] REQUEST_METHOD: `GET` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 14:44:07.818309 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] SCRIPT_NAME: `/identity` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 14:44:07.818578 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80133) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 14:44:07.820266 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.fernet_utils [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80133) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 14:44:07.831052 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 14:44:07.832648 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc0695-3be0-4176-96eb-d8a613f49470 service placement] RBAC: Authorization granted {{(pid=80133) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 14:44:07.842495 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: [pid: 80133|app: 0|req: 1440/2897] 10.222.0.50 () {60 vars in 1365 bytes} [Mon Aug 8 14:44:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 14:44:17.838161 dv-cinder-852009-2-iscsi devstack@keystone.service[80133]: DEBUG dbcounter [-] [80133] Writing DB stats keystone:SELECT=58 {{(pid=80133) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}